In this article, we examine what Big Data is, how it works in practice, and how companies can apply it. This post is part of a series I'm writing on Industry 4.0, in which we answer questions such as: “What is IoT?“, “How do you create a data-driven organization?“, and “What is Industry 4.0?“.

What's Big Data?

In short, it refers to datasets that are too large or complex to be processed, analysed, or utilised using standard methods, according to Oxford Learner’s Dictionaries.

Big Data is based on digitally available data, involving quantities, characters, or symbols processed, stored, transmitted as electrical signals, and recorded on magnetic, optical, or mechanical media.

What Types of Big Data Can We Distinguish?

Big Data can be divided into three types: structured, unstructured, and semi-structured data.

Structured Big Data

Structured data refers to any data that can be stored, accessed, and processed in a predefined format, such as data in database tables. When dealing with large volumes of structured data, we refer to it as structured Big Data.

Unstructured Big Data

Unstructured data lacks a known format or structure. It is not only large in volume but also challenging to process. A typical example is a data source containing plain text files, images, and videos, such as Google search results, which may include text, images, and videos.

Semi-Structured Big Data

Semi-structured Big Data contains both types of data. It refers to data structured in a fixed format but not defined, for example, in a table. An example is data in an XML file whose format needs to be defined in Excel.

What Are the Main Characteristics of Big Data?

Big Data is characterised by the properties of Volume, Variety, Velocity, and Variability.

Volume

The term Big Data derives from the large amount of data. The volume of data is a crucial factor in determining the value of a dataset.

Variety

The variety of Big Data is determined by both the nature of the data (structured/unstructured) and the sources of the data. While sources originally consisted of spreadsheets and databases, the variety of Big Data can be expanded with data in the form of emails, photos, control equipment, PDFs, audio, and so on.

Velocity

Velocity refers to the speed at which data is generated from various sources, such as business processes, application logs, networks, social media sites, sensors, mobile devices, and more. This generates a massive and continuous stream of data at high speed.

How Does Big Data Work in Practice, and How Can It Be Applied in Business?

Here are some practical examples:

Emergency Information

In New York City, 911 emergency information is enhanced with Big Data. Through partnerships with organisations like Apple, Android, and Uber, relevant data from a patient's phone and wearables can be sent to emergency services during crises. This includes GPS location and real-time sensor data, enabling faster and more effective emergency responses.

Netflix – House of Cards

The former hit Netflix series House of Cards was developed using Big Data. Netflix's analysis showed that viewers of the older British series House of Cards also frequently watched films by David Fincher starring Kevin Spacey. Based on this, they predicted that a combination of these three factors would lead to a hit series. Years later, Big Data not only determines which films and series Netflix invests in but also how series are presented to subscribers. Based on viewing history (including points where users pause), it is determined which thumbnails appear on the homepage under "Popular on Netflix."

Skupos: Convenience for Convenience Stores

The US-based Skupos platform collects transaction data from 7,000 different convenience stores. This amounts to billions of transactions per year, which gives retailers insight into determining bestsellers and recommending orders by location. Meanwhile, distributors can predict demand and brands can analyse a constant stream of product data.

Find all our Data Analysis & AI/ML solutions

If you want to unlock the full potential of Big Data, Syndustry is here to help you succeed. With our specialised Data Analytse services and AI/ML technologies, we provide the expertise you need to gain valuable insights and get the most out of your Big Data efforts.

Wondering if your organisation could benefit from Big Data?