Big data is a term for data so large or complex that traditional processing applications are inadequate. Challenges include:
- capture, data-curation & data-management,
- analysis, search & querying,
- sharing, storage & transfer,
- visualization, and
- information privacy.
The term often refers simply to the use of predictive analytics or certain other advanced methods to extract value from data, and seldom to a particular size of data set. Accuracy in big data may lead to more confident decision making, and better decisions can result in greater operational efficiency, cost reduction and reduced risk.
At StreamHPC we’re focused on optimizing (predictive) analytic and data-handling software, as these tend to be slow. We solved Big Data problems at two aspects: real-time pre-processing (filtering, structuring, etc) and analytics (including in-memory search on a GPU).
