if you haven't read the factorio devblog before, please do!
after HN, it's one of my favourite places on the internet because i constantly learn new, random, insane things almost every time. imho it teaches you how to think + shows you what great engineering taste looks like. sorry if i'm overly effusive but each post is so deeply technical and well-written that i can't believe it's free.
you don't need to know anything about factorio or gamedev btw (i don't), just pick a random number between 1 to 438 and start reading :)
I used to say the same about the Eve Online dev blog, but at some point I stopped reading because they were creating an ad hoc, informally-specified, bug-ridden, slow implementation of half of Erlang.
Concurrency is hard. Blizzard added progressively more and more concurrency over time to rescue orphaned resources assigned to a single shard that was undersubscribed while another shard in the same AZ was seeing flash mobs. But the way they documented it was more of a tea leaves situation. Only enough data to guess what they had done if you were familiar with the space.
One of the most interesting things to me when reading this was that it was treated as a bug even though it was that hard to reproduce. Most dev shops would not have the bandwidth and management but in to spend the time to dig into something like that unless it was high severity, and also it sounds like it was also getting caused from a modded version of the software
What’s interesting to me about Prof Berger’s work is that while many of his ideas are seemingly powerful and should be very impactful, it’s underutilized in practice. I think that’s because he’s failed to figure out how to really make it easy to use and productized. For example, Coz should be the definitive tool and mechanism people use to do performance analysis. However because it’s so hard to use and fails in various ways and is barely maintained if at all it ends up seeing limited use.
It’s a shame. The real world of development would be significantly richer if these ideas had better funding and dedicated long term development.
> DTHREADS works by exploding multithreaded applications into multiple processes, with private, copy-on-write mappings to shared memory... Experimental results show that DTHREADS substantially outperforms a state-of-the-art deterministic runtime system, and for a majority of the benchmarks evaluated here, matches and occasionally exceeds the performance of pthreads.
if you haven't read the factorio devblog before, please do!
after HN, it's one of my favourite places on the internet because i constantly learn new, random, insane things almost every time. imho it teaches you how to think + shows you what great engineering taste looks like. sorry if i'm overly effusive but each post is so deeply technical and well-written that i can't believe it's free.
you don't need to know anything about factorio or gamedev btw (i don't), just pick a random number between 1 to 438 and start reading :)
Yep FFF is such a treat to read. They do an amazing job of explaining complex problems and their solution in a way where anyone can understand.
I used to say the same about the Eve Online dev blog, but at some point I stopped reading because they were creating an ad hoc, informally-specified, bug-ridden, slow implementation of half of Erlang.
Concurrency is hard. Blizzard added progressively more and more concurrency over time to rescue orphaned resources assigned to a single shard that was undersubscribed while another shard in the same AZ was seeing flash mobs. But the way they documented it was more of a tea leaves situation. Only enough data to guess what they had done if you were familiar with the space.
One of the most interesting things to me when reading this was that it was treated as a bug even though it was that hard to reproduce. Most dev shops would not have the bandwidth and management but in to spend the time to dig into something like that unless it was high severity, and also it sounds like it was also getting caused from a modded version of the software
My favorite paper on it doing a clever workaround:
https://github.com/emeryberger/dthreads
What’s interesting to me about Prof Berger’s work is that while many of his ideas are seemingly powerful and should be very impactful, it’s underutilized in practice. I think that’s because he’s failed to figure out how to really make it easy to use and productized. For example, Coz should be the definitive tool and mechanism people use to do performance analysis. However because it’s so hard to use and fails in various ways and is barely maintained if at all it ends up seeing limited use.
It’s a shame. The real world of development would be significantly richer if these ideas had better funding and dedicated long term development.
Paper link (2011): https://people.cs.umass.edu/~emery/pubs/dthreads-sosp11.pdf
> DTHREADS works by exploding multithreaded applications into multiple processes, with private, copy-on-write mappings to shared memory... Experimental results show that DTHREADS substantially outperforms a state-of-the-art deterministic runtime system, and for a majority of the benchmarks evaluated here, matches and occasionally exceeds the performance of pthreads.