The Benefits of Big Data and Forecasting in Today's Economy

Forecasting is a prediction of some future event. It is used as a mechanism for strategic planning in most industries but is pronounced in marketing, economics, human resources and investing that rely on future predictions to determine the best courses of actions. Forecasting affords the ability to understand market risks as companies seek to stay ahead of changes and adjust their processes to meet market challenges.

Forecasting can be simple or it can be complex. Typically methodologies use formulas and strict processes to ensure they are giving fair weight and evaluation to the necessary factors used in making that prediction. The process is so important that people have made their living simply off of analyzing information and making usable intelligence for others.

Predictions are as much likely to be wrong as they are to be correct. The problem with predictions is that no one can truly know the exact possibility of the future or all of the outlier events that can change a course of action. However, it is possible that people can determine within a range the most likely events based upon today's influencing factors.

 How and what we see in big data makes a big difference on the quality of the prediction. Some methodologies funnel certain information into the prediction model leaving out clues that may also have relevance. Other times big data can confuse practitioners and they are unable to find any meaningful patterns in the information.

San Diego is getting up to speed on using big data to not only solve problems and make better scientific predictions. According to an article in UT San Diego biomedical community is hiring big data and software gurus to work on problems that range from cancer to Alzheimer. The collection and analysis of information can find statistical significant among events leading to better prevention of future health problems. 

Forecasting and predictions rely on the analysis of big data. Without the ability to understand past data and put it within an appropriate mental framework the patterns of past behavior cannot be used to predict future behavior. Ensuring that you have the capacity to collect information, analyze that information, and put it to good use is furthered through scientific thinking and better data management systems.

Events in Isolation: One way to evaluate likely actions is to assess the probability of events occurring not in a sequence but as stand alone events. The history of the particular problem doesn't matter as long as you have the data to understand how each factor influences the changes one route will happen over another.

Consider how the flip of a coin will either land on a head or tail regardless of previous spins. We know this is 50%/50%. Other events may be 30%/70% among two possible outcomes or 10%, 40%, 50% among three possible outcomes. Each possible outcome has an associated probability of it occurring and can be used in the prediction model.

The problem is that many people do not always see all of the outcomes or know the possible probable outcomes. Their mind and perspective skips over important data that is useful in the analysis. For example, one person may see two possible outcomes and another three. If you cannot see all the possible outcomes then your percentages are likely to be skewed making an assessment incorrect.

Events Based on History: Events don't always work in isolation and have historical outcomes. Understanding history and they way in which things turned out in the past helps to determine your current probabilities. Those probabilities are used to make an evaluation of the current situation to determine whether or not a particular outcome is likely.

For example, if we looked back at the history of a dog sniffing the grass in front of your house you may need to watch what other dogs, as well as that dog, have done over the course of a period of time. If the dog sniffs the same place 70% of the time it walks past the chances are it will do it again. This determination would not be possible without a historical evaluation.

Events Based on Trends and Momentum: Sometimes events are already in a sequence of actions and simply taking a snap shop of it like that used in a Markov chain will not produce an accurate forecast. Evaluating the trend of an action will help determine what influences are likely to either maintain its current trajectory or influence a changes in that trajectory. A ball in motion may stay in motion.

Finding trends means looking at information not only from its current place but also its longitudinal history. This often requires multiple methods of making measurements to see how things are changing. For example, over the past two years unemployment is dropping and assuming nothing in substantial in the market is changing then that trajectory will continue to move forward until other economic factors slow it down.

Events Based on Human Goal Directed Behavior: Humans have animal spirits that are led by logic, emotion, and social expectations to come to conclusion about certain events. Their natural selection and backgrounds will determine what they see in their environment and how they will evaluate that information. Their behaviors are a result of their conclusions and choices will likely be directed toward their goals.

Understanding the history of the person, what type of information they have, the medium they received it in, and the messages create a better interpretation of the individual and their likely conclusions. Such behavior is often calculated through polls, sociological/psychological studies, focus groups and marketing analysis.

Robbins, G. (2015). UCSD hiring 'big data' stars. UT San Diego. Retrieved
http://www.utsandiego.com/news/2015/apr/15/bigdata-stars-ucsd/

Forecasting The Markets by Managing the Data Monster

Accurate forecasting data is paramount to successfully trending the market and creating strategic advantages. A paper by Bill Stringer discusses the use of big data programs and how this influences the success of the chemical industry. For the purposes of my own research, I am considering the merits of big data in proper market forecasting. 

Big data can uncover information that is not easy to find or discern. The relationship between the data pieces offers an opportunity to find trends and information that is not contained in standalone measurements.  When organizations use big data well they can more accurately assess performance and variability to meet market trends. 

The author believes that big data is the next frontier of nearly every economic sector. Companies will find ways to appropriately analyze large and complex data sets in both the private and public arenas. The ability to harness the data monster will allow for greater waves of growth, productivity and innovation. 

Most of the world’s electronic data has been generated in the past few years and executives have yet to come to grips with the virtual world. As organizations get better at analytics they will be able to make more informed decisions. Using data to create better pricing strategies helps in providing greater profitability.

Market forecasting is complex and often relies on experience and the overall feeling of the analyst. This makes traditional methods relatively inaccurate. Larger data can afford the possibility of finding future demand and trends in order to encourage higher levels of organizational growth. 

The paper targets Dow as a primary big data leader. In fostering their growth they recruited 10 PhDs in computer sciences and supported them by a team of advanced analytic experts that mesh their skills with a business intelligence team. Their success is based on a number of important factors:

-Improved accuracy of forecasting. 

-Early indications of targets to correct actions.

-Cost and exchange rate analysis that offers better purchasing of materials.

-Greater staffing abilities that offer better skill levels at the times they are needed. 

The author indicates that data can be drawn from a number of important sources that include social networking and publically available data. Companies can use this data to make a better environmental scan and then incorporate that information to encourage stronger decision making. As companies improve in their ability to forecast the market, they will be able to stay ahead of market trends. 

The report highlights a few important issues. If we think about how public information encourages or discourages business decisions we will find that in general available pubic data is helpful. The problem is that many cities and states don’t have enough information to encourage investment and reduce risk. Offering better data, even if not analyzed, will allow for greater transparency as well as resources for public consumption for strategic decision making. 

Philinkiene, V. (2008). Competitive market demand: the conception and types in the context of forecasting. Economics & Management.

Market Research and Market Forecast

Understanding the needs of the customer is a fundamental activity that coincides with the development of the business. As customers and their patronage are the lifeblood of an organizations existence it is extremely important for organizations understand their needs and wants. One way to do this is to determine the market trends and attempt to find products and solutions that will appeal to customers. To do this well would require a level of market research and market forecasting.

The American Marketing Association defines market research as, “the systematic gathering, recording, and analyzing of data with respect to a particular market, where market refers to a specific customer group in a specific geographic area” (2011).  Market research collects specific information to study market characteristics while a marketing analysis puts the information into a framework that is understood for prediction purposes.

Some assessment tools, such as Ansoff’s Matrix Analysis, provides for a systematic analysis of four general classes product/market growth strategies which include 1.) market penetration; 2) market development; 3) product development; 4) and diversification (Finch, 2012). Even though the analysis is beneficial one must go into greater detail and research to understand these concepts to make them useful.

Market Penetration: Increasing current sales to existing markets.

Market Development: Increasing sales by selling existing products in new markets.

Product Development: Selling additional products to current customers.

Diversification Growth: Selling new products in new markets to create diversification.

Each strategy has its own particular benefits and detractors. For example, if a market is currently saturated it may not be beneficial to create additional market penetration. Likewise, sometimes it can be beneficial to diversify products and markets when traditional products and services are on the decline. To know when each strategy is likely to be successful requires the ability to forecast the market properly.  Inaccurate or misaligned strategies can cost companies their competitive abilities and possibly their future sustainability.

When one has useful information and has analyzed trends it is possible to forecast the market. In general, forecasting the environment should be completed in the context of competitive market demand of both internal and external factors within the market (Pilinkiene, 2009). This means that one should understand the trends, available products, and even the company’s internal abilities to determine whether or not its offerings are competitive.

The constantly changing market requires a new way of thinking about forecasting. As most forecasting models use mathematical models they often ignore the qualitative aspects of the market and therefore become stale (Pilinkienu, 2008).  It is necessary to understand that older models may be limiting in terms of their accuracy. Relying too heavily on one strategy, means that one is not getting the full market picture. 

When organizations do not take into account the full spectrum of relevant information the market has to offer or take in multiple methods of evaluating that market there is a high potential of poor judgment. Executives cannot make proper decisions unless proper information and analysis is conducted. Ensuring that analysis are thorough take into account varying circumstances and offer contingency plans can be helpful in the decision making process.

 Companies should seek to reevaluate their marketing strategies at least once every five years with yearly adjustments. Adjustment periods depend on the type of industry one is in. For example, traditional services have a longer projection time than fast paced technology bubbles. In fast paced markets it may be necessary to develop a marketing plan for each product or service and continue to adjust it as needed or new information arises.

American Marketing Association. (2011). Retrieved from http://www.marketingpower .com/_layouts/Dictionary.aspx

Finch, J. (2012). Managerial marketing. San Diego, CA: Bridgepoint Education, Inc.

Pilinkiene, V. (2009). Forecasting environment and its factors when assessing the competitive market demand. Economics & Management, p. 878-883. 

Pilinkienu, V. (2008). Market demand forecasting models and their elements in the context of competitive market. Engineering Economics, 60 (2).