Project Tungsten is a new thing in the Spark world. As we all know, Spark is taking over the big data landscape. But as always happens in the big data space, what Spark could do a year ago is radically different from what Spark can do today. It busted the big data sort benchmark last year, and is just getting better as it goes. A project called Tungsten represents a huge leap forward for Spark, particularly in the area of performance. But, being me, I wanted to know what Tungsten was, how it worked, and why it improved Spark performance so much.
Big Data Page by Paige
Thoughts on Analytics, Software and Data Management
Archive for in-chip tag
In an article written last year by an industry analyst that I respect, IDC’s Carl Olofson, he gave the impression that in-memory analytics are the wave of the future, the new paradigm for high performance analytic databases. He said, “embrace the new paradigm and plan for it.”
For once, I didn’t agree with him.
In-memory analytics are last decade’s revolution, or even last century’s. The wave of the future is something far faster, and far more revolutionary.