Data Science, It's a Cultural Thing

'It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.'

- Sherlock Holmes,  A Scandal in Bohemia

I’ve always loved this quote from Sherlock Holmes.  I used to battle this every day (much less now) when I was starting my data analytics journey.  Old assumptions die hard.  Just because it happened before, or we observed a fact, it didn’t mean that was the answer forever.  New data and new facts can change assumptions, and if we’re lucky, change our direction if we can spot it early enough.

Luckily I’m several years into my data journey, and don’t have to press (as much) when I say we need to look at the data.  I believe it’s because I’m at the center of an organization that has successfully endured a cultural shift in regards to understanding the importance of interpreting the data, using it to make good decisions.

I recently came across a study published by IBM's Center for Applied Insights titled: Breakthrough experiments in data science: Practical lessons for success.  I wish they had published this when I began my journey, it might have made it a bit less bumpy.  I’ve decided to look at the main points and give my comments on the practical advice from the researchers. 

1.    Infuse data science into the culture

This wasn’t easy, but I agree is of paramount importance.  Your organization has to come to a point where the trust in the data is essential because they use it to make decisions for their business.  Having the mindset that “how will we capture the data on this new process so we can learn from it?” will take you a long way to success.

2.    Design a data science capability

     

     This seems like a no-brainer, but it is so important to success. How are you capturing data?  What sorts of governance have you applied?  Have you started with the end in mind?  Data is so valuable, and so important to good outcomes, that having a good foundation program capability will set you up for later breakthroughs.

3.    Equip with the right technology

I love spreadsheets.  I cut my analytics teeth using Excel (okay, Lotus 1-2-3) and various add-ins. They work well, but aren’t the best tool in all cases.  They require a lot from the user in order to manipulate the data for the desired outcome.  I’ve found that statistical tools like SPSS and Watson Analytics can do a lot of heavy lifting when trying to analyze data without a lot of (comparatively speaking) work.

4.    Showcase your results

Who doesn’t like to say “I told you so?”  Maybe not the best attitude to win over converts, but you get the idea.  There are so many anecdotal examples that companies run for long times on that it becomes something not to question.  One thing I’ve learned is that big prizes are to be won when we question orthodoxy.  What better way to do it than with data?  People don’t know what they don’t know.  It’s the data scientist’s job to do just that.

 Link to Breakthrough experiments in data science: Practical lessons for success

About to start a genuine Watson Project

I talk about Watson l lot.  So far, except for using Watson Analytics for structured data analysis, I haven't actually used Watson or any of the API's for unstructured data analysis.

I've seen them.  

I've read about them.  

I've studied them.

But I haven't used them.  

All that is about to change.  Very soon I'm embarking on a true "cognitive" project.  Its both exciting and frightening at the same time.  

Exciting because I LOVE new technologies, especially ones that change our world.  Watson's ability to analyze paragraphs of text and return answers to questions, learning as it goes along, is nothing short of amazing.  

Frightening because it represents change.  Unknown change.  I read articles every day about robots taking our jobs, replacing the functions we humans perform every day.  

I've reconciled my thoughts by realizing that Watson can help us do what we don't do well, and support us in the things we do well.   I cant quote war and piece, but Watson can do it for me.  Watson  can hold a conversation, but it doesn't have the charm and ability to connect with people the way I do.

Time will tell how this works out, but for one thing, it will be an amazing ride.

Who I'll be reporting to, if I'm not replaced

Analytics in the Cognitive Era

IBM did an excellent job keeping me busy this week, not that I mind at all.  I spoke to analysts, customers, gave presentations, and was included in a keynote session with IBM Analytics GM Alistair Rennie during the general session on Watson Analytics and Cognos Analytics.

I've been in the analytics space for a very long time, and in regards to IBM have seen the platform evolve over the last decade or so.  What's interesting to note is that not all of the progressions were instant hits (powerplay to analysis studio anyone?) hits, but they all eventually became great products that increased the business intelligence of organizations across the globe.

Here's a rundown of what I was asked to input on as customer and a thought leader.  Fast forward the link to 1:08 to get to the introduction and hear my thoughts on analytics in the cognitive era.

Link to day 2 at IBM Insight

Me and Alistair Rennie, GM of IBM Analytics

The Cognitive Era

IBM’s CEO, Ginni Rometty declared this week that we are in the “Cognitive Era”.  According to IBM’s release, we are in a new era of technology, a new era of business, a new era of thinking. What does this all mean?  How can we apply it?

To give some context to the cognitive era we have to look at the business computer “eras” that have come before this.  The “Tabulation Era” was the first.  Calculating and accounting for all of the transactions that occurred were used by super computers to keep track of the statistics.  Issuing inventory was kept in a ledger, and then the data was entered into the GL systems for tracing and accounting.

The second era came with the advances in technology to automate processes, or the “Automation Era”.  Think of computer programming and the programs that combined multiple functions together.  ERP systems combined operations processes such as inventory issuance with accounting systems eliminating the need for multiple processes in order to record the transactions.  The system is “rules based” and rigid based on what was programmed.

That brings us to what is being described as the “Cognitive Era”.  Dictionary.com defines “Cognitive” as “relating to the mental processes of perception, memory, judgment and reasoning”.  So the “cognitive era” in our example means that the systems will use all of the information available to manage the supply chain of inventory as it is required based on cognition.  The difference here is that the computer “learns” the system, and makes decisions based on what it concludes, much like a human does. 

So, what does this mean for the new era?  According to IBM, 80% of all data is invisible to computers.  The data they are referring to is “unstructured data”, which are paragraphs of text.  The other 20% has been the subject of what they call “big data”, a term used for all of the massive amounts of data that is being created every time we turn on our smart phones, post a tweet on Twitter, or enter a credit card number for a purchase on Amazon.com.  Decisions across the globe are made every second based on analytics arising from the 20% of structured data that’s analyzed.  Combine these decision paths with the 80% of structured data into a system that “thinks” and you’ll get better outcomes to very difficult problems.

I was recently given a powerful example of the combination of structured and unstructured data in decision making that will change how quickly a company can respond to changing product demands.   In this scenario, a marketer searches keyword terms (unstructured data) on a popular upcoming kid’s movie across all social media.  The keywords are analyzed by the system in a cognitive fashion for the most popular terms associated with the original terms.  That is, the system “thinks ahead” for the marketer using all of the unstructured data at its disposal.  It also refines and learns patters in the data to come up with new techniques in understanding that is being searched for.  The unstructured data is combined with sales data, location information and demographics (structured data) to determine where the product is most needed, by whom, and when.  Data is transformed cognitively into actionable information, in just a few clicks. 

The real difference here from a typical program is that the learning piece is added to the equation.  For computers to think and learn patterns from data is not new, in my opinion.  What is trans formative into this new area is the cognitive system to process all of the data, structured and unstructured, at the velocity and volumes that are being created today.  It’s not just making a decision on the available data that makes this era so unique.  It’s making decisions using ALL of the data.

I once was fortunate enough to have lunch with several others with Steve Mills, Executive Vice President, IBM Software and Systems back in 2011.  He was asked a question by one of the people attending, “How do we know what data to capture?” His answer, in light of this cognitive era was poignant 4 years ago.  “All of it” he answered.

I think he was right.

Hosting a Dinner with Chef Watson

I recently hosted a business dinner to discuss cognitive technologies.  What better way to do this than serve a creation by Chef Watson?  So I gave instructions: this will be a plated dinner, make what you want, BUT you muse serve one course created by Chef Watson, which can be accessed by anyone at https://www.ibmchefwatson.com/.  I was so proud of myself for thinking of this idea, I couldn't contain it.

On the day of the dinner, I was mentioning my proud surprise to a colleague, and was asked "What if the recipe doesn't taste good?"

Ever have one of those moments where you thought you'd thought of everything, but in reality you forgot what might be a fatal flaw?  I became a bit nervous at that moment, to describe it in the least.

When I arrived early to set up materials for dinner, this is what greeted me:

After seeing this I felt a lot better.  After eating this below, I even felt better than that.

Chef Watson eats data, not food, so my concern was valid.  But I'm reminded that there are correlations in data I still haven't found and are waiting to be explored.  We live in exciting times for people that love data.

Now I'm wondering when the Watson App for working off Chef Watson creations will be available on Bluemix?

Tools That Help Me Love Data

I'm a data geek.  A data nerd.  I think about data and I think about how to analyze data.  So much so, that my kids are starting to pick up my habit, or at least answer me in a logical, structured way.

Driving to school yesterday with my 10 year old daughter London,  we started discussing her upcoming research project.  Her school has a project that is done in groups every year from first grade on, and now she is at the level where she gets to do one by herself, on a topic that means a lot to her.

There are 3 things she likes, and one of them is math.  Numbers.  She loves them.  Stats, she understands them.  Calculations, she does them for fun.  You get the idea.

The question she wants to research: "Are girls better at math than boys?"

I stepped lightly into this (potential) minefield  of a discussion, mind you, and began the Socratic method.

Dad:"What do you think?"
London:"I don't know."
D:"How could you find out?"
L:"I could ask a bunch of girls and a bunch of boys."
D:"Would that be objective, or subjective?"
L:"Probably subjective"
D:"How could you get objective data?"

This went on until we determined there are probably some data sets somewhere she can get to analyze the performance of students in math.  Google becomes your best friend in these situations, right?  The world is full of data waiting to be analyzed. Data that's been collected that is just that, collected data.  What we need is to turn that data into information we can use to communicate our finding.

This when got me thinking of another problem: How am I going to teach data analysis to a (smart) 10 year old?  Something I've spent my career learning to effectively do?  Where do I start? Spreadsheets?  A stats program like SPSS?

London is lucky in a few ways.  1.  She is good at asking questions. 2.  She knows how to use a web browser.  3.  She can upload a file.

Because of this, her research analysis just became a lot easier (for both of us).  I helped London create her own IBM ID, the only other prerequisite other than the 3 above that is needed to get started immediately with Watson Analytics.  The videos explain step by step how to upload data and begin analysis right away.  The best part?  She can use her inquisitive mind to ask Watson questions and Watson will give her answers. It's so easy, so easy that a 10 year old can get value out of using cognitive technologies to learn.

Structured vs Unstructured Data Analysis

Reviewing the landscape

Watson technology is exciting.  IBM has made is easy for developers to access artificial intelligence tools to add to their designs through Bluemix, a platform-as-a-service offering from IBM.  What does this mean?  It means quite simply that anyone can use watson and develop Watson solutions.

Some of my favorites are there...Q&A, Personality Insights, and speech recognition.  Others, though, seem to be missing.  Where is Watson Analytics?  The existing Watson APIs on Bluemix do a great job handling unstructured (paragraphs of text) but missing is the power to analyze structured data (database type).

To analyze structured data, there is a platform of Watson Analytics that allows anyone to find correlations in their data for free.  The issue I have at this time is that they are both under the Watson banner, but don't work together in the way that all other Watson products work together. Watson Analytics is a suite of analytics tools that include natural language processing options, but aren't available as an API to integrate the 2 types of data in one query.

Hopefully we will see an addition to the Watson API options soon.