Blending Brandwatch data with Apache Drill

Using Brandwatch, you can collect some fascinating data. Our clients use our platform to gain intelligence on what the world is saying about what matters to them most. Often, we notice that users will export data from the app and then compare and contrast it with other types of data like their sales figures. Sometimes this can be a tricky process, requiring a lot of Excel wizardry or worse: manual editing, cutting and pasting! Recently, in my Funky Friday time, I was experimenting with Apache Drill, which has just announced its 1.0 release. Drill is a tool for doing »

Scaling code review

Consider two tables. I'm not talking about database tables here; I'm talking about items of furniture. The first table is a flat-pack effort from a popular high-street retailer. The second is an artisan mahogany table with legs turned in a beautiful fluted Sheraton style. Which would you choose? Well, it's clear that both of the items serve the same purpose as a raised surface to place items, but it's obvious that the latter is better: it has better build quality, superior materials, and you'd wager that you would pass it down through many generations to come. The first one will »

Scaling: thinking in hardware and software

I remember learning about hardware when I was a Computer Science undergraduate. It was mostly taught separately from software with both subjects residing in different modules on the curriculum. To students without much industry experience, it can be easy to think that hardware and software are extremely different subject areas; the former populated with stuffy electronic engineers and silicon, and the latter being where the real computer science happens: data structures, algorithms and code. At Brandwatch, we find that the best backend developers are the ones that think in hardware as much as software. We have very data-centric technology on »

Quaid, Start the Reactor!

This post was originally published on Brandwatch's main blog on June 16th 2014. We are the Reactor team. Unfortunately we do not look after nuclear reactors (no matter how much we dream) although we do have 3 Ph.D. engineers on the team. What do we do? Well, Reactor is one of the key use-cases that the Engineering team wants to build software for. A Reactor wants to know the latest and most important information right now. They don’t care about noise. They’re busy people, and they want to react to a crisis before it happens, not afterward. »

More and More Data: How We’re Confronting the Challenges

This post was originally published on Brandwatch's main blog on 1st August 2014. Working at Brandwatch over the past three years has been exciting. As a world-leading social media monitoring tool, we crawl the web in real-time for mentions that match customers’ queries. There are various facets to the challenges we face, the first being that our business is booming. As mentioned on a previous blog post by our Architecture Lead Steve Mason, the number of client queries is doubling every year. In addition to this, the amount of data that we crawl is always increasing. With news articles, forum »