Author: Julien Grossmann, Political and Violent Risk Data Analyst, HIS Economic and Country Risk
I’ve been using KNIME Analytics Platform for a year and a half, and in this time, KNIME has become a vital part of my work. As a political and violent risks data scientist, I am often confronted with incomplete or badly structured data. But with KNIME, I can always find ways to efficiently clean up, organize and analyze my data, and this, despite a total lack of programming or coding knowledge.
One of the most frustrating aspects of working with unstructured or semi structured data, is that they are rarely ready made for your needs. Therefore, they always require extensive manual clean up.
Worse, if your needs are constantly shifting, the data restructuration and clean up are too.
For instance, the typical data I work with involve datasets of events (such as civil unrest or terrorism attacks). Most data sets have basic meta data like date, location, and a short description of the incident, but there is only so much you can conclude from such basic dataset. Often I need to drill down, and look for specific groups, or specific actions or targets.
In the old days, we would use basic search functions in excel, or for the less geeky of us, we would do it manually.
So I decided to create a little workflow using the KNIME Textprocessing extension. The idea was to be able to mine large datasets rapidly, using a customizable list of keywords.