Skip to content

Big Data & AI - Where People and Answers Live

Source: https://www.wte.net/Big-Data-AI-Where-People-and-Answers-Live
Date: November 2022
Author: Eric Garrison

Introduction

Garrison discusses foundational mega-trends in technology, particularly how cloud computing enables responsive websites, digital marketing, and custom software solutions. His LinkedIn post about embracing big data sparked significant interest, prompting this comprehensive exploration of data analytics and artificial intelligence.

What is Big Data?

Oracle defines big data as data that contains greater variety, arriving in increasing volumes and with ever-higher velocity (characterized by volume, velocity, variety, and veracity). Garrison proposes that big data represents the volume of information needed to generate actionable insights.

A practical example: a modest 200-page website with 6,000 customers and 300 daily visitors generates approximately 700,000 data points annually. Over five years, this grows to nearly five million data points—demonstrating how data expansion outpaces customer growth at a 3–5x rate.

Big Data's Customer Challenge

Garrison warns against losing sight of customers while pursuing data analysis. He cites Forbes contributor Aaron Spinley, who emphasizes that "the best advertising is not in the data; it's in the creative..." Spinley advocates focusing on behavioral and psychological factors rather than purely data-driven segmentation.

The article cautions against "MarTech hubris"—overconfidence in predictive analytics—referencing Nassim Taleb's Black Swan concept. The key principle: technology should serve customer understanding, not become an end unto itself.

What is Artificial Intelligence?

IBM's definition frames AI as leveraging computers to "mimic the problem-solving and decision-making capabilities of the human mind." AI encompasses machine learning and deep learning, enabling systems to make predictions based on input data.

Practical examples include:

  • Customer service chatbots (Zendesk)
  • Writing assistants (Grammarly, Wordtune, ProWritingAid)
  • Domino's Pizza Tracker—using algorithms to manage customer expectations through data visualization

Big Data and AI Tools

Analytics Tools

  • Google Analytics (transitioning to GA4)
  • Matomo
  • Smartlook
  • Clicky (privacy-focused alternative)

Big Data Platforms

  • Hadoop Ecosystem
  • HPCC
  • Apache Cassandra

No-Code Analytics Solutions

  • DOMO—rapid dashboard creation
  • Tableau—drag-and-drop visualization with VizQL
  • Microsoft Power BI—machine learning integration (WTE is a licensed reseller)
  • Polymer—spreadsheet-to-database conversion

Customers - Where Answers Live

Two key recommendations:

1. Asking for Help: Crowdsourcing solutions builds collective intelligence and customer engagement through public credit and transparency.

2. Sharing Behind-the-Scenes Content: Revealing creative processes (design, writing, analysis) builds authenticity and trust. Customers understand the infrastructure exists; transparency differentiates brands.

Big Data Terms for Technical Readers

  • Hadoop: Open-source framework processing gigabytes to petabytes efficiently
  • Data Lake: Central repository storing unstructured data of any type
  • MapReduce: Software framework processing multi-terabyte datasets in parallel
  • HDFS: Primary Hadoop storage system
  • HBase: NoSQL distributed datastore enabling real-time petabyte access
  • Fault Tolerance: System continuity despite component failures
  • Spark: Distributed processing system combining with R/Python for pattern detection
  • Pig/Pig Latin: High-level abstraction tools above MapReduce
  • Data Abstraction: Simplifying complex datasets into essential elements
  • OOP: Organizing software around data objects rather than functions
  • Hive: Hadoop-based framework processing petabytes

Closing Question

Garrison invites reader feedback on favorite big data and analytics tools, offering to feature recommendations with attribution.

Key Takeaways

  1. Every organization generates big data; growth requires automated management
  2. Customer-centric thinking must guide data strategy
  3. AI and machine learning amplify insights when focused on human behavior
  4. No-code analytics tools democratize data visualization
  5. Transparency and community engagement outperform data opacity