Posted by For some time now, big data has been a controversial topic. While its defenders highlight the great usefulness of big data in different areas, its detractors express doubts regarding the protection of people’s privacy. Edward Snowden’s controversial statements regarding espionage and surveillance practices carried out globally have only fueled the fire of general distrust regarding the protection of citizens’ personal data. In this general atmosphere of suspicion, everything that revolves around big data comes loaded with negative connotations although, in reality, this aspect is only part of the definition of this concept.
What is big data?
This term, or its equivalent “massive data”, refers to a volume of data so complex that it cannot be processed with any type of traditional software and hardware. In itself, it is a neutral concept, since it can also refer to an endless amount of data from research. However, given that the data collected may be of a personal nature (users’ communication or consumption habits), the term has ended up taking on a negative tone. Its detractors see in the collection of data and, especially, in its evaluation, a clear attack on personal rights.
How big is big data?
The concept of big data does not refer to a specific amount of data, since there is no defined limitation from which massive data becomes considered big data. In practice, the term is synonymous with a volume of data that cannot be measured in gigabytes.
How is big data generated?
The volume of data has grown exponentially: the amount of data collected by humanity from the beginning of its history until 2002 was generated in 2014 in a period of ten minutes. According to forecasts, this mountain of data will continue to grow, doubling every two years. This flood of data is a consequence of general digitalization in all areas of daily life and comes from sources such as:
- Internet connection from mobile
- Social networks
- Geolocation
- Cloud Computing
- Vital data measurement
- Consumption of audiovisual media
Big data does not only refer to the data itself but also to its analysis and use. In this evaluation process, we try to look for patterns and connections to contextualize them correctly. The challenge is not only the enormous volume of data, but also its speed and variety (the three “Vs” of big data), since these constantly go to an unstructured file and must be, ideally, recorded, stored and processed in real time. To read them correctly and be able to connect them, a sophisticated data infrastructure is necessary.
How can I work with big data?
Big data is responsible for the emergence of new technical requirements regarding custom software development services. To analyze this data, special frameworks (digital infrastructures) are required, whose main function is to process as many data sets as possible and import them quickly. Additionally, this software must make data available to the user in real time and respond to multiple database requests at the same time.
Hadoop is a well-known open source solution for this purpose, although its extremely complex implementation often requires support from experts, and so-called data scientists. However, to enter the world of big data, cloud solutions are a very good option.
Examples of use of big data
The areas of application of big data include various aspects of daily life. For example, in the eCommerce sector, anyone who has ever purchased from an online store knows the phrase “customers who bought product A also bought product B.” These recommendations are based on the evaluation of millions of data from other customers.
Other areas of application include :
- Medical Research – By evaluating big data, scientists can find the best therapeutic solutions and plans for their patients.
- Industry: By using data about their machines, custom software development company can increase productivity and work more sustainably.
- Economy: Big data allows companies to know their customers better and better adjust their offers.
- Energy: to adapt energy consumption to individual needs, you must know them. The collection of consumption data allows for a sustainable energy supply.
- Marketing – Big data in marketing is often used for targeting purposes. The purpose is, above all, to improve customer relations and increase conversions in different strategies.
- Fight against crime: also for government and state security, the collection and analysis of massive data is useful (for example, to combat terrorism).
Criticisms of big data
Mainly it is about the protection of people’s privacy. When it comes to optimizing marketing strategies, large databases present great potential for companies and brands, since, used to segment their audience according to their behavior, they serve to create much more precise user profiles. And this is considered in itself an attack on the user’s private sphere. That is why if you are dealing with big data, you must inform the website (Data Protection section) about the use of personal data collected from users.
Another critical point is “ data autocracy ”. Naturally, in this area, there are also the so-called “big players”, companies that, for years, have been working with large databases and obtaining benefits from them (Google and other search engines). This is how a few companies such as software development services monopolize the ownership of that data. And this is precisely what is criticized, as well as the danger of large-scale data abuse. Without the existence of clear rules regarding data protection and the anonymity of the evaluated data, the possibility of misuse of user data cannot be excluded.
Responsible use of big data
Assuming correct use of technology, at the other extreme are the benefits that the use of big data can provide. For example, some important advances in cancer research would not have been possible without the use of big data. Power supply and traffic control are also continuously optimized by evaluating massive data, offering greater safety in daily life.
Despite all the opportunities it offers in medicine, traffic control and economics, ethical questions arise regarding big data, since, for example, the prediction of certain events, such as the probability that someone will be a victim of a certain disease, remains certainly undesirable for many. Citizens have their reservations in this regard, causing fear of information collection companies to grow more and more. In this sense, experts advocate for transparency when informing users about the use of their data and for the creation of a legal framework that allows technological advances without losing sight of the protection of people’s privacy.