All posts by Sudipta Choudhury

Sudipta Choudhury is a Technical / Business Writer with considerable industry experience in various domains. He can be reached at

To Hadoop or Not

Refer “Big Data” or “Data Analytics” today and you would definitely hear of “Hadoop”. Hadoop is often positioned as “The Framework” which would solve all your Big Data needs. This framework from Apache Software Foundation is an Open Source Framework, i.e. anyone can use the framework for free and is capable of handling huge data sets across a large set of commodity hardware. Therefore, it is popular among the development community.

Big Data, from its very nomenclature refers to data which is huge, structured or unstructured. Examples of such data are the enormous amounts of data in social media sites, evolved due to interaction of the fraternity of social media users of Facebook, Twitter, LinkedIn, etc on a day to day basis. Types of such data include the various types of chat information, images, videos, etc which are being used by the users of such applications. Other applications providing such data are from iOT applications like data related to processes in industries or manufacturing units such as temperature, pressure, etc. that keep on changing over real time and result in huge data sets, if we measure the data over certain periods of time. Other such data could be data related to telecom usage, space or weather data, stock trading data, etc.

Hadoop is handy if your needs are that of ETL (Extract – Transform and Load) Operations. However, do not get into the Hadoop trap unless you have a clear understanding of your business needs. Ask yourself the following questions before you decide on investing your time and money implementing on Hadoop.

  • Do you really have terabytes or petabytes of data to be processed? Hadoop was designed to handle huge data volumes of this scale. However, a report from Microsoft states that majority of jobs process less than 100 GB of data. If your data size is lower than terabytes, you may not require a Hadoop. Even if your data size is more than terabytes, do you really need to process all your data?
  • Are your data needs real time? If your expectations of processing data are real time, Hadoop is not the best tool to meet your needs. In fact, Hadoop requires some time to process data and is a very good batch processing tool. In case your business is to interpret data in real time, such as movement of data in the stock markets for taking real time decisions such as buying or selling stocks, Hadoop is not the answer.
  • Do you require a quick response? Your requirements for response time need to be well understood. If your user is not interested to wait for a minute to look at response for large data sets, you may have to use other real time applications and not Hadoop.
  • Does your requirement involve complex and computation intensive algorithms? The alogorithm of MapReduce in Hadoop is efficient in handling processing of large volume of data on a parallel processing mode by dividing large files into smaller files and storing across machines. However, this is not apt for requirements which are computation intensive and having large number of intermediate steps of data during computation, e.g. the computation of Fibonacci Series. A few machine learning algorithms also do not fall in the paradigm of MapReduce and therefore the pertinent question here to decide is whether the business requires high usage of specialised algorithms. In that case, the technology experts need to analyze if the algorithms required are MapReducible.

Hadoop would be your choice when:

  • You want to transform largely unstructured or semi-structured data into usable form.
  • You want to analyze information of a huge set of data to obtain insights, and you have ample time for such analysis.
  • Have overnight batch jobs to process, e.g. daily transactions processing in case of credit card companies
  • When the insight gained from such data analysis is applicable over a longer period of time, e.g. social behavior analysis in case of social sites, e.g likes and affinity analysis, job suggestions based on browsing history, etc.
  • Hadoop utilizes key value pairs during its processing efficiently, and such forms of data are ideally useful for its operations.

So… before you rush in to select Hadoop as your framework, analyze your needs carefully. Though Hadoop is a free framework, its implementation might require effort and cost and the budget for implementation may not be that cheap.

Why Dave, the Salesman Hates the CRM

It is quite common in organizations. A new director joins the sales team, looks for marketing and sales details of performance and cannot find those. The only data that he has is the sales per team member, the targets set and the performance achieved by the sales team. Sales team members work in isolation and fear to share data even with their own team leaders. The organization seems to be “performing”, and in an “auto-driven mode”. Good enough…isn’t it?

Not so for the director, who thinks how he can comprehend the overall performance without “reliable and sufficient data”.  How can he control the process? What is the solution to this?

The Nirvana does not seem to be far off. The director’s ray of hope hinges around a tool based on Customer Relationship Management (CRM).  Customer Relationship Management has a wide connotation and means a set of technologies, practices, and strategies for companies to analyze or manage interactions and data during the entire life cycle of customer. By this process of customer relationship, customer retention is improved ultimately helping sales growth. A CRM package can provide automation in various flavors such as Sales Force Automation, Marketing Automation and Services Automation.  A collaborative CRM ideally can establish integration with external suppliers, distributors and vendors. Besides, it may be highly Analytical, offering data mining, pattern recognition and correlation features.

The director is convinced, and soon a CRM is implemented across the organization. Dave, the Salesman is not convinced with this approach. The sales team is used to manual systems and adopting a CRM results in a set of challenges:

  • The Sales team members usually like interactions with people, rather than with a CRM. Their time is well spent talking to potential customers, rather than entering data.
  • Usage of CRM apparently does not benefit  the Sales team members in achieving their targets. The CRM disrupts their existing modes of working, which, they believe have helped them to be “successful” in achieving targets.
  • Lack of systems / IT knowledge among the sales team members  cause hindrance to adoption.
  • The decision for CRM implementation was taken by Senior Management without involving the rest of business or the users of the system. Stakeholders like Dave are bound to be unhappy as their needs are not completely addressed.
  • Over complicated systems require adequate user training and hand holding, followed by self-practice.

The challenges seem to override the obvious benefits that a CRM brings forth, such as creating more visibility in operations and monitoring, storing, archiving and extracting data securely based on access rights of the users.

How do we ensure that organizations adopt CRM effectively and that the implementation does not result in a misadventure? Here are a few tips from the experts:

  • Convey objectives of using CRM clearly to Sales Team. Business benefits for the organization as a whole, and ease of operations for sales team need to be conveyed clearly.
  • Involve the sales team in designing of the system. This would ensure buy-in of the sales team members for using CRM.
  • Demonstrate Return on Investment (ROI) of such a venture, not only for the organization, but also for the sales team. Communicate the wins due to CRM implementation to the team.
  • Finally, a major reason for the frustration of sales folks is the complexity of CRM. Often, the CRM uses data intensive entry screens, making entry of data mandatory. Lot of this data is not required to be entered. Complexity of data needs to be minimized, making it easy to use.

Therefore, any implementation of CRM would require sufficient planning to ensure that the stakeholders are involved, and their requirements are met. Finally, Dave should like the CRM instead of hating it.

Big Data to Bigger Data, Microprocessor to DNA – A Competition with God

Integrated Circuits (ICs) came into existence during the eighties, when there was a need to pack a large number of transistors, diodes, etc.  into a small chip. The 8085s, and 8086s of the world quickly evolved into different ICs, which became ubiquitous by their presence among a wide gamut of computers and customers. Miniaturization, and addition of more power in a chip has been on the rise since then, as the data needs have been constantly increasing.

The world of Big Data has set goals for the next level of chip evolution.  IC manufacturers have been quite aggressive, and competitive in miniaturization, and processing power.  This journey for chip makers, however, is likely to reach a dead end soon with silicon technologies reaching their limits. This is a potential risk for evolving technologies such as Artificial Intelligence, which would require humongous data, or a BIGGER DATA set than Big Data. So…what is the mitigation for this risk??

Scientists have found a solution for such limitation in God’s own creation – the DNA – the natural supercomputers that exist in millions in human bodies. Deoxyribonucleic Acid (i.e.  DNA), which comprises various genes, performs functions, and calculations much faster than the fastest supercomputers on this earth, as of today.

Microsoft, along with a team of researchers from academics recently used Biotechnology to store information of about 200 Megabytes along with a compressed video in a fraction of a liquid drop.  The binary code of 1s and 0s, used in silicon technology was deftly converted, and mapped to the four bases in DNA – Adenine, Cytosine, Thymine, and Guanine by this team.

DNA storage involves some of the latest technologies in security and data compression. This purports to develop a completely new humanoid like another human being, and therefore, can be used extensively while making chips. The units of coding DNA are very small – less than half a nanometer, whereas the modern day transistors are about 10 nanometer in size. Moreover, DNA can be packed in a three dimensional configuration unlike transistor chips, which are packed in a 2 dimensional mode. This results in high processing power for DNA of more than 1000 times over the Silicon chips.

Therefore, DNA would be of great use in storage technologies. We shall be using God’s own creation to derive unprecedented benefits in data based technologies the near future, and potentially create humanoids more powerful than human beings.

Why Are We Scared of Artificial Intelligence

Artificial Intelligence (AI) is being considered as the next technology evolution, which would disrupt a large chunk of jobs performed by human beings. Learning machines, which are being designed, are likely to learn faster than human beings. Consequently, they could even turn out to be our foes, or pose as survival threats to human beings. Not surprising though, a host of important personalities, such as Bill Gates of Microsoft, Elon Musk of Tesla, and Stephen Hawking, the famous physicist of our times, have warned us about the serious consequences that AI would cause to the human race.

The last few years have witnessed lot of work, and implementation in the automation space, particularly in Artificial Intelligence. Without appropriate regulations in place, there could be valid reasons for such fears to be expressed. Think of a scenario where AI is in wrong hands, and is being used with evil intent!!

The stage of AI research today is far below the level that could cause robots to dominate over human beings. Essentially, AI is based on code, or algorithm written by developers. Therefore, the focus of AI being used is limited and focused to specific jobs in hand. This means, the robots with AI are still stupid. They are totally controlled by scientists who design their intelligence. Usage of Artificial Intelligence today is limited to specific activities, which are repetitive in nature. Robots that are designed based on AI focus on a narrow scope of work- they possess Artificial Narrow Intelligence (ANI). Human intelligence, on the contrary, is manifested in multiple tasks, and encompasses a wide assortment of activities that cannot be easily replicable.

Examples of AI could be a driverless car, or IBM Watson’s Jeopardy supercomputer, which could potentially beat the world chess champion, or self-learning bots. How do you think that these objects work? Simple…. They rely on a large set of data – Big Data, a huge number of “ifs and then” programming, i.e. a large number of algorithms designed by human beings, NLP (Natural Language Processing, for their language skills, if any), etc.. If there is a defect, or bug in programming, the computer has to obey the master and, cannot automatically rectify itself. Therefore, the AI systems are not creative enough to build upon themselves. They still depend on their masters who design, and completely control their operations.

Like any other machine, an AI based machine has its own limitations, and could be dangerous if not controlled. Take for instance the recent “driverless car” from Tesla which met with an accident, killing the onboard human driver. Unless the algorithms are properly written in the AI processors, such costly mistakes are bound to happen.

Human beings are blessed with “common sense” that helps in taking decisions easily, whereas these fifth generation machines lack this human attribute. They are essentially “dumb” outside whatever they have “learnt”. They still have to be monitored, mentored, and controlled by human beings. Hence, fears related to apocalypse are not well founded.

As more automation is built in, and around our systems including those with AI, existing jobs would get replaced, or in some cases, updated to extend the AI based tools.  Therefore, introduction of Artificial Intelligence based robots also could replace, or extend some of these jobs. Nevertheless, because of “common sense”, human beings would be better placed than robots to evolve, and take up new roles, which would add value to our society. Time and again, evolution of various technologies have taught us this lesson, and AI would be no different.

Brexit, Automation, Digital Age and Us

The referendum for Brexit by the people of the United Kingdom seeking to part ways with the European Union (EU) throws a few poignant questions on where the world is heading. The instability is further enhanced by visible cues from the Republican Party in the United States with its projected leader, Donald Trump seeking to rake up the xenophobic feelings against migration, and hatred for a religious minority.

  • Are we seeking to go back to our past to resurrect a state, where countries were isolated, and collaborations were limited?
  • In this connected world, where messages move almost at the speed of light, can isolation work?

In the last two decades, the internet has become deeply entrenched, and associated with day to day lives.  Technical advancements have been quick, and human beings in general have gained by having better control, and exposure to usage of services. These advancements in general have affected some of the jobs, but created a whole range of new services and work areas. Information Technology (IT) industry therefore, has, not only created numerous jobs, but also has given ample options for reskilling, and redistribution of labor.

One such advancement of technology which is abuzz is “Robots”. These were created, and used by the Japanese, who were the pioneers in research with Artificial Intelligence. More often, they used these robots as toys, and in games. They did replace a few mundane, and repetitive jobs in manufacturing as part of Flexible Manufacturing Systems, particularly in automobile companies like Toyota. The nature of these displacements, however, was not as much as impacting as the recent experiments suggest. With the U.S & Japanese companies, along with, Research & Development establishments devoting considerable effort to create humanoids and build intelligence, new developments which are termed “disruptive” have emerged, each of  which could potentially remove jobs in millions. Take for instance the “driverless car” experiments from Google or a few auto companies in USA.  What happens to the jobs of drivers? The drones from the US Department of Defense are operational for some time, particularly for reconnaissance, and in war to weed out terrorists without “pilots” onboard.

Replacement of labor in industries with the purpose of gaining operational efficiency, and profitability has been in vogue for quite some time – from the introduction of machines to automate the textile industry; to introduction of automated systems and flexible manufacturing systems; to companies like IBM, and Accenture that have moved to destinations abroad to scout for cheap techies in place of the expensive ones in the United States.

When it comes to robots, however, the cost advantage that they bring in to the table cannot be matched by any salaried employee – in fact robots do not work for salary. They are adept at doing repetitive jobs, at no extra cost without becoming fatigued. Not surprisingly, the world’s three large employers – Foxconn, US Department of Defense, and Walmart are replacing workers with robots, reports Business Insider. Foxconn is a key manufacturer for Google, Apple and Amazon – is 10th largest employer in the world, and has used robots to replace 60,000 workers. Citi and Oxford predict that about 77% jobs in China, and 57% jobs from 34 OECD countries are prone to risks due to automation. The World Economic Forum estimates that by 2020, 5 million jobs could be lost globally.  A utilization of automation dubbed Robotic Process Automation ( RPA) is also being developed and implemented which would potentially replace cheap BPO jobs as well. It is just not the workers who are at risk. Even the highly educated professionals like medical, or journalists could be at risk if artificial Intelligence has its way. In fact, IBM has claimed to develop a computer that can diagnose cancer better than doctors.

It is this undercurrent of technology, coupled with a long recession, which seems to have created a jingoist attitude – that of holding on to whatever available, or grabbing even what is not available among the masses, causing Brexit. It reminds one of famine – the Russian famine, consequently forcing human beings into cannibalism, in order to survive.

The neo-Luddites are back.  The army of textile workers, known as Luddites had protested against the machines introduced during the Industrial Revolution in England, since then and the struggle seems to have continued, albeit in a different way during this Fourth Industrial revolution of Robots, RPA, Artificial Intelligence, Digital Technologies and human beings.

While robots usually create efficiency and profits for the companies they work for, they also require human support – to manage and monitor them, to maintain them and replace them; if the need be. As such, artificial intelligence (AI) is “artificial”, as it is developed completely by humans. Each norm of intelligence that is impregnated within a robot with AI requires development, testing, and implementation by human beings.  These programs also require inputs from data analysts, since any artificial intelligence requires a lot of learning that the robot (humanoid) has to do.  For example, just to make a robot understand what a “flower” is, it has to be fed a lot of comprehensive data. Recently, Google’s artificial intelligence program erred by tagging a Black man as a Gorrilla, due to inadequate data. The vast requirements of data that the Artificial Intelligence, and Digital Technologies require is called Big Data in Software parlance. This in turn requires new software, and technical members to maintain such data and databases. Data Analyst is a new job profile that is created to define, and analyze such data.

Each technological revolution leads to job loss for some, and new jobs created for others. It is important to reskill, and keep oneself updated with skills, which would be applicable under the transformed environment. If the number of jobs created are less than the jobs lost, we are going to see more and more of social adjustments like Brexit or social unrest. Governments need to collaborate more on job creation, since only job creation for human beings would keep the social environment under control.

The future for us is CHANGE, so as to adjust to rapid automation. We need to learn to coexist with the robots and being productive during the age of robots.


Progressive Web Apps – A Promise for Web Developers

“Disruptive” or “Radical” are some adjectives to describe novel technologies. “Ajax”, with its concept of responsiveness was one which caused transition in web engineering rivalling traditional applications on the web. The other transition which was notable was arrival of native apps in the mobility domain. With Android and iPhone aggressively grasping market shares from the traditional blackberry’s, a new demand was witnessed in the year 2014 – that of native apps which could be developed and stored in Google or App stores and used directly by mobile devices. Mobile websites seem to have lost the battle to these apps, which were capturing the market rapidly.  No longer being profitable, these sites were in the process of closing their shutters. By the end of 2015, millions of apps were developed (1.5 million in App Store and another 1.9 million in Google Play Store).

Are all of these apps of use? The answer is “No”. Majority of these apps today are “zombies” – unused apps consuming stores, which have never been downloaded.  Therefore, the question still remains – whether the promise of native apps is fading and eventually would be lost out to the web?

Native apps were “hot potatoes” to users because of their feeling of ownership, of having apps on the device home screens, fast loading and offline usage. Mobile developers were quick to latch on to this demand and developed countless apps. Web developers, on the other hand, focused on server side technologies, new JavaScript features and components – Node.js, Angular.js, React.js, HTML5, Web API, single page applications etc. The developments in this domain were relatively slow, given the limitations of these technologies.  Things however, started to change with the advent of Progressive Web Apps. The functionalities, which were viewed as key advantages of using apps on smartphone, were progressively available for the web developers to extend to their mobile websites.

Progressive web apps (PWAs) use the same set of technologies that developers usually use – HTML, CSS, and JavaScript. Moreover, they do not require special IDEs for development, such as Android Kit or iOS SDKs, but could be developed using simple editors like Notepad++.  With inherent advantages of these apps, web developers are excited again about the numerous possibilities that this technology would create for them.

Still confused with the idea of PWAs??

Here is a definition from Google – “Progressive Web Apps are experiences that combine the best of the web and the best of apps.” Essentially, these apps tend to create similar user experience for the web user, to those of mobile users using native apps. As the name suggests, these Apps offer a few key features:

  • Progressive, in order to accommodate any user irrespective of their choice of browsers.
  • Responsive to any kind of screen – laptop, desktop, mobile, tablet or any other form of screen.
  • Tolerant to Connectivity issues, i.e. can work offline in networks with poor connectivity, with the help of service workers.
  • Like App, i.e. built with an App Shell model, it offers an app like feel to the user on the web with styles like apps, navigation and integrations.
  • Latest updates using service worker updates.
  • Secured using HTTPS protocol so as to avoid pilfering or corruption of information.
  • Search Engine Friendly, ensuring easy identification and accessibility with Search engines such as Google or Bing, etc.
  • Push notifications to reengage customers lost due to non-availability or lost sessions. The Service worker threads, which remain “alive” beyond the life of a browser session, hold the key to these notifications. These workers make the notifications available during the following sessions to the users.
  • Easy availability of Apps on Home Screen without the need for an App Store.
  • Shareable by URLs – no complex installation of apps required.

With the arena for progressive web apps heating up, there are some interesting uses of this concept. FlipKart claims to have used PWAs to design FlipKart Lite that has improved retention of users by about 70%.  The other site which has received rave reviews for these apps is The Washington Post.

Like any other technology, Progressive Web Apps have their own set of issues. A cause for concern is that the mobile web site is turning more app-like creating a demarcation in look, feel and usage between a mobile website and a standard website. For any App to qualify as a PWA on the Google Chrome browser, the browser looks for a few properties and if satisfied, extends a few abilities like the ability to add to a home screen of a smartphone. PWAs also require to provide manifests for their sites, with the display status – “standalone”, “fullscreen”, and “browser”. The “browser” is the only option where the URL can be seen, and the look is like a web browser. However, Google does not register this mode as a PWA.

Therefore, developers seem to be focusing more on the “app” part rather than the “web”.  The URL, which is an essential ingredient of the web since its inception is no more to be seen in these implementations, essentially stripping the web of its basic feature, i.e the address bar.

As usual, the development is chaotic and messy, but that does not deter the developers from aggressively establishing this as a standard. Expectations are high, and very soon, the effort is likely to bring out the best that the web has to offer, eventually making users even stronger.



Is the World a better Place for Travel

Worldwide economic slowdown, conflicts and terrorist attacks, and the European refugee crisis, apparently seem to have their impact on the global travel & tourism industry.  However, when we look at the actual trend, the industry has maintained its growth despite all the adversities. A report from IPK International World Travel Monitor, 2015 reports a 4.5% growth in actual outbound trips in 2015, with a healthy increment of 4.3% estimated for 2016.

Outbound travel is primarily fueled by Asia Pacific and North America.  Germany, as a country is the ‘world travel champion’ – a leader in outbound travel. The United States follows Germany and continues to be both a leading source and destination for travel. China enjoys a leadership position in the travel industry, being next only to USA in terms of spending.

The European economy has improved slightly from 2014, primarily due to the growth of Germany. Overall, the forecast for Europe is a net growth of 2.8% in outbound travelers. While Europeans have maintained their momentum, they are likely to travel to safer destinations, avoiding the zones of conflicts and terrorism. Also, there is good growth in inbound travel to Europe, and the expected growth is between 3 to 4 % in 2016. Travelers from China and Asia Pacific countries, USA and Japan are all keen for travelling to Europe.

Outboubound Travel Forecast Percentage GrowthEconomic growth has slowed down to a certain extent in Asia Pacific, but, despite the slowdown, the number of travelers have only increased and the projected growth of 6.3% for outbound travelers is on expected lines. The growth rate as per IMF report for 2015 shows that India’s rate of growth is highest at 7.6%, ahead of China, and therefore, outbound travelers from and to India are expected to increase.

While North America is expected to show good growth in outbound travel, South America’s 1.9% growth is a cause for concern. About half of South America’s outbound travel market is catered to by Brazil and Argentina. Traditionally, South Americans travel internationally within the same region. One international event which could improve the percentage of international travel to Brazil could be the Olympic Games planned to be hosted in the city of Rio de Janeiro this year. The last Football World Cup in Brazil in the year 2014 caused more than half a million visitors to Brazil, and this trend is expected to be seen during the Olympics.

The Middle East travel market is one of the fastest growing markets and countries like Saudi Arabia and United Arab Emirates (UAE) are leaders in this area. The region is noted for travelers with deep pockets, who usually travel for long durations (with average trips for more than 14 nights). Also, more than 30% of travelers are immigrants travelling to meet friends and relatives. Inbound travel to Middle East has been seriously hurt due to the ongoing conflicts in that zone.

Last, but not the least, social media plays a vital role in international travel today. 70% of international travelers are active users of social media such as Facebook, Twitter, WhatsApp, LinkedIn, Google+. About 30% of the international travelers actively use social media for planning their trips. Marketers would do well to be creative with innovative approaches, so as to influence this section of buyers to plan their trips.

Do Not Let Your Data Kill You – The Need for 3 R’s – Reduce, Recycle and Reuse

As the saying goes – anything in excess is a waste. Isn’t it true for information today?  Information or “data” – the four letter word which is more representative of the digital world has overwhelmed you, me and everyone transcending this space. Data in this form has various connotations – the more popular “Big Data”, Large or complex data, humongous data, etc.

On an average, data of companies have been increasing at a rapid pace – about 100% or more every year. Also, with users of social media being overactive, data transactions have multiplied manifold in real time. Though technical advances are being made to store this data in large repositories, there is a need for deriving context – meaningful information so as to Reduce, Recycle and Reuse data. For example, companies would like to use their data to understand and interpret information such as employee interactions, communications and client engagements. Data that is not used, but occupies useful repository space is a costly waste and needs to be eliminated. Regulatory requirements require one to use data to create intelligent and statutory reports that can be audited easily if the need be. The 3 R’s put in practice improve data management in a business environment:

Reduce:  Regulatory requirements for data, e.g. PCI data storage requirements or other Information governance or compliance standards, require one to be circumspect before planning for reduction of data. This challenge for cleaning up data not only results in a large volume of unused data, but also results in saving of data in local repositories of users with subsequent backups by the IT team.

Therefore, how do I reduce unused data? A Document Retention Policy, specifying the criteria for holding or removing data, the process governing such a decision and the relevant owners to implement and oversee is the first proactive step that any company can adopt that only appropriate data is maintained. With a policy in place, the discipline to actually implement such a policy enables a large reduction in unused data.

Recycle:  Regulatory Reporting is an important aspect for many industries. For example, in the US, Health industry related reports are mandatory, not only for the companies, but also for the patients, and the industry is well regulated.  Taxation or Financial obligations also require statutory reporting and audits. It is important for the data to be recycled and processed into useful reports for the auditors and the statutory authorities. Usually, intelligent software, ETL techniques, help in recycling such data.

Reuse: The most interesting part of data management is Reuse of data. The world of Business Analytics and Business Intelligence has offered options for deriving business insights from a large data set and intelligently reuse data. A new science “Data Science” has evolved in its own right and is promptly advocated by the Harvard Business Review. The HBR article from Thomas H Davenport and D J Patil in fact refers the job of a data scientist as the “sexiest job of 21st century”.

A few terms often used for reuse of data are:

  • Data Science: This is a term which loosely entails the combo of computer science, analytics, statistics, and data modeling. While this is a loose combination, and some companies have evolved their own courses or certifications, it still needs to mature as a science with comprehensive tenets and elaborate literature.
  • Smart data: Smart data is usually a subset of Big Data, with noise filtered out. While Big Data can be characterized by its attributes – variety, velocity and volume, a smart data is usually is characterized by velocity and value. Smart data is a key ingredient for intelligent BI Reporting.
  • Predictive Analytics: It involves smart methodologies utilizing data – machine learning techniques and statistical algorithms to predict the future outcomes of data. Companies gain out of predictive analytics by deriving or planning important outcomes from past data, e.g. revenue or profit.
  • Real Time Analytics: Analytics served real time, e.g. stock prices moving up or down, updates on page views, sessions, bounce rates, page navigation, advertisements dynamically adjusted based on type and frequency of customer usage, etc.
  • Intelligent Decision Systems: Use of Artificial intelligence in association with data is an area that helps users to derive the best and optimized decisions based on a large number of input variables. While this is still evolving, it can be used in number of areas such as building marketing systems that offer customers based on profile analysis, blocking of fraudulent transactions in credit card operations, etc.
  • Data Visualization: Pictorial or graphical representation of data intelligently, in an interactive way, help business professionals to identify trends and patterns in their data, e.g. sales data region-wise, or by customer profile.
  • Big Data Analytics: Reuse of data is not complete unless we use the term Big Data. The concept of Big data analytics has evolved from companies managing huge sets of data such as oil companies or telecommunication companies to social media such as Facebook, Twitter, LinkedIn that involve large data sets. This form of analytics help us to derive hidden patterns, market trends, preferences of customers, unknown correlations, etc.

 Business Data Analytics, therefore is in its infancy, to be nurtured, developed and evolved over the years. The attraction therefore is immense, and so is the job of the Data Scientist!!!