Magazine article Information Today

Mitigating the Challenges of Big Data

Magazine article Information Today

Mitigating the Challenges of Big Data

Article excerpt

Big Data has multiple definitions depending on the context, ranging from large or complex datasets to sophisticated technology suites that capture, store, manipulate, and analyze large volumes of data. But the usage of Big Data and its value to an organization is consistent: Big Data maximizes the value of your data by applying analytics to drive smart decisions.

Regardless of the definition, the ultimate value of Big Data is constrained by the veracity and comprehensiveness of the underlying datasets. Typically, Big Data is composed of large volumes of variable data from disparate systems and content repositories. Merging the data in a meaningful manner and extracting the relevant data points must be done carefully to accurately represent the whole. What's the best approach to accomplish this? It all comes down to a well-considered, rigorously planned and managed conversion process.

Whether you're working in technology, writing an article such as this one, or simply taking a drive, having a general idea of your destination will go a long way toward reducing detours and dead ends. Likewise, with Big Data, organizations need to first understand which data is essential to strategically direct their business. They can then intelligently assess the diverse data sources across their organization and the benefits inherent in each. This is important because, as noted, the ultimate function of your data--Big Data--is to derive insights that define and achieve corporate strategies and goals. This idea goes out the window if you don't identify and consolidate the right data from the beginning. And equally so if you don't understand your data and its origin, variety, and intent. Even if you have the "right" data, but it's incomplete, that too could be problematic--your analytics may be shallow, and your results may be skewed.

Assess Your Data

In the simplest terms, you need to assess what data is important and viable versus what data is irrelevant, inconsistent, or contradictory. Focus on the main sources of each dataset and the relationships between the datasets, then iteratively classify your data and map it to your business taxonomy. The data-mapping process is a helpful tool to diagnose flaws in the underlying data and highlight opportunities to improve your data quality, which is often accomplished by robust conversion methodologies that leverage cross-data validation techniques. Comparable data frequently resides in multiple sources and formats, both internal and external to an organization. Matching, validating, and enriching similar data across different sources may enable you to significantly improve the quality of your Big Data and resulting analytics.

Creating a comprehensive corporate taxonomy is a critical step in aligning your data with your corporate strategy. To achieve this, you need to do the following:

* Classify your data within your taxonomy.

* Determine the right format for your Big Data to ensure its sustainability and viability.

* Study the source data and find the right level of granularity, so that the stream of data continually ingested into your model is effectivity applied.

Remember, all of this data will be used for Big Data analytics, and if you understand how it was created, the meaning of each element, and its intent and quality, you will get closer to meeting your goals. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.