How to increase uthash bucket size to accomodate a large number of values

[ahm_irf]

I am using uthash for hashing and indexing a large number of values.

I wrote a program which has possible power(256,4) = 4294967296 values to be indexed. But uthash could not manage to indexed all those values. It could not increase its bucket size from 16777216.

Could anyone please tell me how can I increase the bucket size to accommodate a big number of values like 4294967296 values.

[wje_lf]

It could not increase its bucket size

With hash functions, a "bucket" is the set of data items with the same hash value. Instead, I think you mean the total number of items in the hash.

And the problem here is that you're hashing a huge number of items. You probably ran out of virtual memory.

If you really want to hash that many items, you need to do two things.

1. Run on a 64-bit machine, because even with just one byte per item, you're running out of address space with a 32-bit machine.
2. Increase greatly the size of your swap space. Wouldn't hurt to have gobs of RAM also.

If you want to sort that many items and you don't want to do these things, don't hash them like that. Put them in a file and run sort. And bring some knitting to do while it's running.

Hope this helps.