Comments on: Need fine-grained concurrency primitives

By: Eager lazy languages « Handwaving

Eager lazy languages « Handwaving — Fri, 14 Mar 2008 17:44:28 +0000

[...] great thing about the Internet is that CS luminaries might wander by your little blog and tell you you’re wrong. I thought there might be a way to exploit laziness to make more efficient use of the memory [...]

By: Pinku

Pinku — Mon, 10 Mar 2008 22:40:44 +0000

Moon is right: I’m still thinking about squeezing extra speed from conventional RAM models. But conventional hardware is going to be around for quite a while, so it’s not in vain. I have no idea what next-gen hardware might do. My last rambling comment about lazy languages was motivated by my dim memory of the FFTW compiler. FWIW, I’ll explain in a full post.

By: Moon

Moon — Sun, 09 Mar 2008 20:22:10 +0000

Fine-grained concurrency will just advance the computation to the next cache miss faster. I fail to see how that would help! Seems like Amdahl’s law applies here.

Aggressive reordering to avoid fetching the same data into the cache more than once makes a little more sense.

But you are still thinking in terms of an asymptotically smart compiler somehow being able to preserve the illusion of random access memory. My point is that as that becomes increasingly untenable, we will instead be forced to change the model of computation in which the programmer thinks to a model that more closely corresponds to what the hardware can actually deliver. Then the compiler is no longer being asked to perform an impossible job. Unfortunately I do not have any great ideas to offer for how to do this. You could think about modelling memory as a set of streams of data rather than as a random-access array of data.

By: Daniel Weinreb

Daniel Weinreb — Wed, 20 Feb 2008 11:24:58 +0000

Actually, Moon didn’t say anything about fine-grain concurrency control. It is possible to take advantage of multi-core processors with deep caches by simply assigning disjoint main memory to each core and running threads in each core that don’t interact with the others using the sharing of memory; or, at least, share memory in limited and restricted ways, to keep the concurrency control simple.

On the other hand, there has been a lot of theoretical work on CPU’s with built-in transactional capacity at the instruction set level, which would make concurrent programming simpler to think about and do correctly.

I think that as the number of cores keeps going up, there will be a lot of experimentation with different approaches to programming to keep those cores busy. It should be very interesting to see how it all plays out.