Agency asks for Decooda’s help in delivering high-value insights to Fortune 200 Client

by David Johnson on January 3, 2012

After my last blog post ( ), several people asked me to explain more about the process we went through to perform the social media Country of Origin analysis we did for “BigCo” (de-identified Fortune 200 company).  Following is the explanation of the process and some ideas on how the process could be improved.

This was our second experience trying to solve a complex client challenge using another social media monitoring and text analysis company’s data (the first was with a large telecommunications firm).  Ultimately, we were very successful. Unfortunately, the process was overly complicated and time consuming, and the problem is that if we wanted to run the analysis again using the clients’ social media data providers’ data, we’d have to go through the same process…all over again.  However, there is really good news here. I believe this one experience opens the door to a new way to approach social media analytics that is more streamlined and cost effective.

The Client Challenge

“BigCo” wants to be able to identify the majority of Canadian social media posts among 5 million North American posts (Canadian and U.S. co-mingled posts) over a 2.5-year period.  The goal is to improve their competitiveness and increase sales by better understanding “Canadian” insights. This starts by correlating social media discourse attributes to market dynamics in order to intelligently refine tactics and strategies.  ”BigCo’s” AOR (agency of record) asked Decooda if they could do a proof-of-concept to prove/disprove the feasibility of being able to identify Canadian texts with an accuracy rate of at least 70%.  Note: It is believed that success with such a nuanced analysis will also be a good leading indicator for additional efforts in ethnicity and other types of demographic analysis.

The Process

We needed two sets of data to perform the analysis.  First, the AOR provided us with a general set of tens-of-thousands of North American social media posts that we would use as our test dataset to identify “Canadian” authored posts.  Apparently, the client’s current social media analytics vendor’s systems didn’t make it easy to export data considering it came in several individual files.  Second, the AOR provided us with another several thousand posts that could be validated as “Canadian” posts, what we call “gold-standard” data.  The gold-standard data was primarily from Canadian branded pages.  Therefore, we have a high degree of confidence these posts were, in fact, of “Canadian” origin.  The Decooda linguistics and cognitive science team then created linguistic models using the gold-standard data.

Once we had the two data sets, we needed to prep the data so it could be analyzed linguistically.  In this case, preparing the data means removing any remaining HTML artifacts, duplicates, injected ads, parsing the data and marking it up so it can be analyzed.  (Note: Companies that already have a social media analytics vendor are always surprised to find that their data needs additional cleaning, when to the human eye it looks “clean.”)

Once the data was prepared and the model (what will become an analytics Plug-in) was completed, we simply ran the analysis. We finished up with a validation process to test our accuracy, and our work was complete.

The Results

To test linguistic model accuracy, the team co-mingled gold-standard data with the general set of social media data and ran the analysis in order to assess their ability to appropriately tag “Canadian” posts.  The team then used the derived linguistic model to automatically identify Canadian posts, and voila, 76% accurate in identifying Canadian posts and 86% accurate in identifying US posts, approximately 82% accurate in total.

Rethinking the Approach

Although we were successful in performing this analysis, the process is far from streamlined.  We understand why many of the current social media and text analytics vendors might not want to let clients download an unlimited amount of data.  People may abuse access, and the pipe required to allow for the movement of huge amounts of data is extremely expensive. However, we think there is an alternative approach that will streamline the process, lowering risk and improving the client experience.

Decooda’s recommended approach would be to run the analytics against social media posts as they stream into the Decooda Platform in real-time.  As the data streams in, all appropriate sentiment and text analytics would be run on each post, and the measurements (metadata), along with the actual post data would be stored in an indexed database for filtered retrieval.   Further, as new social media data streams in, it too would be analyzed in the same manner, in real-time and stored in the database.

With the metrics now readily available, agencies and brand teams will not need to download the post data to perform an analysis outside of the system.  To get access to these metrics, the client will simply identify the relevant documents through a semantic filtering tool and download the metrics that have been appended to the data. Then, if required, the client or agency can run a deeper regression and cross-tab analysis on the data when combined with other transactional data (sales, loyalty, promotion, etc.).

Now, you might be wondering, what if we don’t have a metric the client wants. Simple. The client, their agency or Decooda can create a new analytics algorithm, what we call a Plug-in, and add it to the Decooda Plug-in Class Library.  Then we point the relevant data to the new Plug-in and append the new resulting metric to the data…again, one simple streamlines process.

The Results

Leveraging the Decooda approach, once the Plug-ins have been run against the data, the client just needs to press a button to receive the metrics.  All data can be exported as a CSV file, Excel file or the data could be streamed directly into an enterprise system.  Poof…all of the data downloading and data manipulation work goes away.   This simple, streamlined process will change the game for brand teams and agencies by offering a high-quality, real-time experience that will lead to better insights, lower operational costs and significantly improved client satisfaction.

However, we also realize that not every company is going to immediately switch to Decooda.  So in order to prove ourselves to companies using competitive social media and text analytics solutions, stay tuned for a solution that will allow Decooda to add value to virtually any client workflow…regardless of what social media and text analytics vendor they are using.

The Benefits

There are many benefits associated with this approach.  However, what stands out in my mind is that Decooda provides the only platform that is capable of addressing both the data mining and discovery process needs that so many agencies and brands are struggling with, while also addressing the client “listening and monitoring agenda”…all at web-scale and at real-time.

In addition, Decooda is not limited to social media data. Clients want to simultaneously analyze public social media data, private social media data and other enterprise data, including transactional data.  Our approach is feed independent, so we can bring it all together for you.

Call Decooda if you want to add value to your existing or new social media analysis process.  You will finally get what you want, when you want it, at an affordable price.

Happy New Year!

David Johnson




Leave a Comment


Previous post:

Next post: