At the recent Data Day Texas event, I sat down with Davin Potts, who I have known for many years, and had a long conversation about a wide variety of subjects. Over on the Vertica blog, I broke the conversation into chunks, but I wanted to put it all together in one place so you can see what we chatted about end to end. So, here’s all of it, from machine learning to open source, from Python to Knime, and why the heck DO we move data out of a database to analyze it?
Big Data Page by Paige
Thoughts on Analytics, Software and Data Management
Archive for KNIME tag
The theme of Data Day TX 2019 was the highly cooperative landscape between proprietary and open source, and a good architect doesn’t choose sides.
I learned a lot at Data Day Texas. I live tweeted a lot of interesting bits on @RobertsPaige as I went along, but some of the most enjoyable and enlightening stuff happened at the happy hour afterward.
In Hadoop’s ecosystem of massively parallel cluster computing frameworks, Actian DataFlow is an anomaly. It’s a powerful little engine that thinks it can take on any data processing problem, no matter the scale. The trouble is that unlike MapReduce, Tez, Spark, Storm and all of the other Hadoop engines, DataFlow is proprietary, not open source.
I just got back from a little executive summit conference in Dallas for Chief Data Officers. Frustratingly, I heard a lot of folks telling me what Hadoop CAN’T do. Now, I know that Hadoop can’t bring about world peace or get my husband to put the toilet seat down, but the things people keep saying it can’t do are things that I’ve personally DONE on Hadoop clusters, so I know they’re doable.
If you asked most people if water could cut through steel, they would probably tell you it can’t. They would be wrong, too.
All of the people at Big Data Tech Con were clearly invested in improving their skills to help address the gap between the business need for big data skills and the people who can meet that need. I proposed that there were two ways that analytics software could help.