-
Data Migration: Understanding the Challenges
Data migration – that is, the practice of sharing and distributing information between databases – requires some very careful consideration. Are you moving the data permanently, temporarily, sharing it between applications? Do want to share all of it, some of it? Are you changing databases, or trying to move some data to access or use the…
-
Process home monitoring data using the Time Series Database in Bluemix
I keep a lot of information about my house – I have had sensors and recording units in various parts of my house years, recording info through a variety of different devices. Over the years I’ve built a number of different solutions for storing and displaying the information, and when the opportunity came up to…
-
Making Real-Time Analytics a Reality — TDWI -The Data Warehousing Institute
My article on how to make the real-time processing of information from traditional transactional stores into Hadoop a reality has been published over at TDWI: Making Real-Time Analytics a Reality — TDWI -The Data Warehousing Institute.
-
Real-Time Data Movement: The Key to Enabling Live Analytics With Hadoop
An article about moving data into Hadoop in real-time has just been published over at DBTA, written by me and my CEO Robert Hodges. In the article I talk about one of the major issues for all people deploying databases in the modern heterogenous world – how do we move and migrate data effectively between entirely…
-
Harvest machine data using Hadoop and Hive
A new article on has been published on IBM developerWorks, looking at the basics of processing machine data using Hadoop, from extracting the core data, storing it, and then determining the baselines and trigger points required to identifying worrying trends and points. From the intro: Machine data can come in many different formats and quantities.…
-
MySQL to Hadoop Step-By-Step
We had a great webinar on Thursday about replicating from MySQL to Hadoop (watch the whole thing). It was great, but one of the questions at the end was ‘is there an easy way to test’. Sadly we can’t go giving out convenient ready-to-run downloads of these things because of licensing and and other complexities,…
-
Real-Time Replication from MySQL to Cassandra
Earlier this month I blogged about our new Hadoop applier, I published the docs for that this week (http://docs.continuent.com/tungsten-replicator-3.0/deployment-hadoop.html) as part of the Tungsten Replicator 3.0 documentation (http://docs.continuent.com/tungsten-replicator-3.0/index.html). It contains some additional interesting nuggets that will appear in future blog posts. The main part of that functionality that performs the actual applier for Hadoop is…
-
Process complex text for information mining
My latest article on data mining text information is now available: Text — an everyday component of nearly all social interaction, social networks, and social sites — is difficult to process. Even the basic task of picking out specific words, phrases, or ideas is challenging. String searches and regex tools don\’t suffice. But the Annotation…
-
Building flexible apps from big data sources
My article on how to build flexible apps on top of the BigInsights platform has been published. This demonstrates a cool way to combine some client-end JavaScript and existing technologies to build a Big Data query interface without developing a specialised application for the purpose. It’s no secret that a significant proportion of the needs…
-
Process big data with Big SQL in InfoSphere BigInsights
The ability to write an SQL statement against your Big Data stored in Hadoop provides some much needed flexibility. Sure, using Hive or HBase you can perform some of those operations, but there are other alternatives that may suit your needs better, such as the Big SQL utility. My latest article on this tool is…