Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The problem is that in most systems you cannot completely avoid shared state. We usually move the responsibility to manage shared state into the DBMS, but the DBMS has to be written in some programming language as well.

As the amount of available RAM grows, in-memory architectures become more desirable, especially for analytics workloads. But how do you do in-memory analytics without shared state? It just doesn't scale.

What we need is a way to make access to shared state explicit, not the default. We also need pausless garbage collection or optional garbage collection so that large amounts of shared in-memory data doesn't cause latency problems.

Pauseless garbage collection conditioned on never using shared state is not good enough for the kind of systems I'm talking about.



This is exactly what ETS in Erlang provides :)


As I understand it, ETS is itself an in-memory database that comes with its own data model, but it doesn't let me build one out of my own data structures and algorithms, which is what I had in mind.

Also, I think, all data has to be copied in and out of ETS. It cannot be referenced in-place, which is going to slow things down quite a bit. (Correct me if I'm wrong. I have no first hand experience with ETS)


Cache misses between disparate CPU cores will also slow you down. Sharing memory is not without its efficiency drawbacks as well.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: