Introducing
Your new presentation assistant.
Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.
Trending searches
Information processing
6
Information processing
four main stages of processing information in an organization
Four main stages of processing information
acquisition or retrieval
Subtopic 1
storage
transformation
transmission
What is needed to enable effective information processing?
Effective information processing
channels of communication must be supported by at least two additional resources: technology infrastructure and management policies. In the modem business climate, it's hard to overstate the value of an efficient information technology infrastructure. This includes everything from being able to store and retrieve information on computers to being able to disseminate information to others through the use of e-mail, corporate intranets, extranets, and other means. In their own right, computers have always been used to process raw data and routine information into more meaningful forms, and this ability is growing ever more sophisticated through the use of expert systems, neural networks, data mining programs, and other advanced software applications. Ultimately, however, high-level information must be processed by humans, and therefore management policies and practices must be in place to facilitate the flow of appropriate information, and in some cases, to discourage the flow of inappropriate information.
EXAMPLE.
EXAMPLE.
If the company wishes to maintain a liberal customer satisfaction policy, customer service representatives must be trained not to refuse customers who want to return merchandise under unusual circumstances not be specifically documented in training manuals. If the corporate objective is to always provide amicable, hassle-free customer service, these employees must be taught the broad customer service philosophy and probably should be empowered to evaluate specific circumstances using that basic knowledge. In other words, assuming that generous customer service is consistent with the business strategy, information processing in this case probably should involve disseminating to employees broad objectives that they then can process themselves and apply to specific instances—a targeted decentralization of information processing.
Four components of the information processing framework
These components specify the structure of an information processing system, whether human or machine
sensors
Subtopic 1
memory
processing mechanisms
access mechanisms
Data mining- definition
Data mining
Also called knowledge discovery in databases. It is the process of discovering interesting and useful patterns and relationships in large volumes of data. The field combines tools from statistics and artificial intelligence with database management to analyze large digital collections, known as data sets.
As computer storage capacities increased during the 1980s, many companies began to store more transactional data. The resulting record collections, often called data warehouses, were too large to be analyzed with traditional statistical approaches. Considerations about how recent advances in the field of artificial intelligence could be adapted for knowledge discovery led in 1995 to the First International Conference on Knowledge Discovery and Data Mining. This was also the period when many early data-mining companies were formed and products were introduced. One of the earliest successful applications of data mining, perhaps second only to marketing research, was credit-card-fraud detection.
origins
Model creation
The complete data-mining process involves multiple steps, from understanding the goals of a project and what data are available to implementing process changes based on the final analysis. The three key computational steps are the model-learning process, model evaluation, and use of the model.
Model learning occurs when one algorithm is applied to data about which the group (or class) attribute is known in order to produce a classifier, or an algorithm learned from the data.
If the model is sufficiently accurate, it can be used to classify data for which the target attribute is unknown.
modeling
- Predictive modeling- is used when the goal is to estimate the value of a particular target attribute and there exist sample training data for which values of that attribute are known.
- Descriptive modeling- also divides data into groups. With clustering, however, the proper groups are not known in advance; the patterns discovered by analyzing the data are used to determine the groups.
- Pattern mining- concentrates on identifying rules that describe specific patterns within the data. Market-basket analysis, which identifies items that typically occur together in purchase transactions, was one of the first applications of data mining.
- Anomaly detection- can be viewed as the flip side of clustering—that is, finding data instances that are unusual and do not fit any established pattern. Fraud detection is an example of anomaly detection.
approaches
The potential for invasion of privacy using data mining has been a concern for many people. Commercial databases may contain detailed records of people’s medical history, purchase transactions, and telephone usage, among other aspects of their lives. Civil libertarians consider some databases held by businesses and governments to be an unwarranted intrusion and an invitation to abuse. Often the risk is not from data mining itself (which usually aims to produce general knowledge rather than to learn information about specific issues) but from misuse or inappropriate disclosure of information in these databases.
privacy concerns
Text mining
Text mining
Definition
Applications
Comparison to data mining
Definition
Text mining, also known as text data mining is the process of deriving high-quality information. The process usually involves the process of structuring the input text, deriving patterns within the structured data, and finally evaluation and interpretation of the output.
Definition
~Wikipedia
Text mining is the set of processes required to turn unstructured text documents or resources into valuable structured information.
~expertsystem.com
Applications
Text mining process is now often applied to a wide variety of government, research, and business needs.
Applications
Security applications
Many text mining software packages are marketed for security applications, especially monitoring and analysis of online plain text sources such as Internet news, blogs, etc. for national security purposes. It is also involved in the study of text encryption/decryption.
Security applications
Biomedical applications
Biomedical applications
A range of text mining applications in the biomedical literature has been described, including computational approaches to assist with studies in protein docking,protein interactions, and protein-disease associations.In addition, with large patient textual datasets in the clinical field, datasets of demographic information in population studies and adverse event reports, text mining can facilitate clinical studies and precision medicine. Text mining algorithms can facilitate the stratification and indexing of specific clinical events in large patient textual datasets of symptoms, side effects, and comorbidities from electronic health records, event reports, and reports from specific diagnostic tests.
Software applications
Text mining methods and software is also being researched and developed by major firms, including IBM and Microsoft, to further automate the mining and analysis processes, and by different firms working in the area of search and indexing in general as a way to improve their results.
Online media applications
Text mining is being used by large media companies, such as the Tribune Company, to clarify information and to provide readers with greater search experiences, which in turn increases site "stickiness" and revenue. Additionally, on the back end, editors are benefiting by being able to share, associate and package news across properties, significantly increasing opportunities to monetize content.
Business and marketing applications
Business and marketing applications
Text mining is starting to be used in marketing as well, more specifically in analytical customer relationship management. Text mining is also being applied in stock returns prediction.
Sentiment analysis
Sentiment analysis may involve analysis of movie reviews for estimating how favorable a review is for a movie. Text has been used to detect emotions in the related area of affective computing. Text based approaches to affective computing have been used on multiple corpora such as students evaluations, children stories and news stories.
Scientific literature mining and academic applications
The issue of text mining is of importance to publishers who hold large databases of information needing indexing for retrieval. This is especially true in scientific disciplines, in which highly specific information is often contained within written text.
Comparison to data mining
Technology perception
Deployment time
Data mining has been considered a proven, robust and industrial technology for many decades.
Data mining is focused on data-dependent activities such as accounting, purchasing, supply chain, CRM, etc. The required data is easy to access and homogeneous. Once algorithms are defined, the solution can be quickly deployed.
Comparison to data mining
Text mining was historically thought of as complex, domain-specific, language-specific, sensitive, experimental, etc. In other words, text mining was not understood well enough to have management support and therefore, was never valued as a ‘must-have’.
The complexity of the data processed make text mining projects longer to deploy. Text mining counts several intermediary linguistic stages of analysis before it can enrich content. Next, relevant terms extraction and metadata association steps tackle structuring the unstructured content to nurture domain-specific applications.
Process mining
Process mining is a relatively young research discipline that sits between computational intelligence and data mining on the one hand, and process modeling and analysis on the
other hand.
Process mining
Scope
Scope
Starting point for process mining is an event log. All process mining techniques assume that it is possible to sequentially record events such that each event refers to an activity and is related to a particular case (i.e., a process instance).
Guiding principles
The IEEE Task Force on Process Mining recently released a manifesto describing guiding principles and challenges
The manifesto aims to increase the visibility of process mining as a new tool to improve the (re)design, control, and support of operational business processes.
It is intended to guide software developers, scientists, consultants, and end users. As an introduction to the state-of-the-art in process
mining, we briefly summarize the main findings reported in the manifesto
The six guiding principles
Process mining is an important tool for modern organizations that need to manage non-trivial operational processes.
Challenges
On the one hand, there is an incredible growth of event data.
On the other hand, processes and information need to be aligned perfectly in order to meet requirements related to compliance, efficiency, and customer service.
Challenges
11 challenges
Business Intelligence
BI is a term that refers to a variety of software applications used
to analyze an organization’s raw data.
It consists of several related activities including data mining, online analytical processing, querying and reporting.
It helps to improve decision making, cutting the costs and identify new business opportunities
Implementation
Although BI applications may come very handy, they usually are also very complex. That is why executives have to ensure that the data feeding BI applications is clean and consistent so that users trust it.
This is a role for the executives to first analyze the needs and then choose the proper way of implementation of the BI.
There are 7 steps that may be used to enroll the BI system
7 steps
1. Make sure your data is clean.
2. Train users effectively.
3. Deploy quickly, then adjust as you go. Don't spend a huge amount of time up
front developing the "perfect" reports because needs will evolve as the
business evolves. Deliver reports that provide the most value quickly, and
then tweak them.
4. Take an integrated approach to building your data warehouse from the
beginning. Make sure you're not locking yourself into an unworkable data
strategy further down the road.
5. Define ROI clearly before you start. Outline the specific benefits you expect
to achieve, then do a reality check every quarter or six months.
6. Focus on business objectives.
7. Don't buy business intelligence software because you think you need it.
Deploy BI with the idea that there are numbers out there that you need to
find, and know roughly where they might be.
Potential problems
Since the BI sometimes may be very complex, the greatest barrier in implementing the new solutions may be the resistance of users.
The users should also keep their data in good working order as data is the core of each information system.
The other issue is that due to the first potential problem, the BI tools have become more for reporting (which is more user-friendly) rather than for business management
Potential problems
4.
BI is pretty new field of interest, and because of that. some companies may not be aware of their process managements well enough to utilize the BI itself. First you need to understand what is your process about in order to improve it.
It is very important to recognize if the process has the direct impact on revenue, to ensure easy data flow and to train the users before the BI implementation
Business Intelligence benefits
Benefits
Business intelligence has been used to identify cost-cutting ideas, uncover business opportunities, roll ERP data into accessible reports, react quickly to retail demand and optimize prices.
BI also may give companies more leverage during negotiations by providing more information about relationships with clients.
It also helps with money management in the company by optimizing business processes and focusing decisions.
Business Intelligence - summary
In order to get the BI right:
• Analyze how executives make decisions.
• Consider what information executives need in order to facilitate quick, accurate
decisions.
• Pay attention to data quality.
• Devise performance metrics that are most relevant to the business.
• Provide the context that influences performance metrics.
Other information
Online analytical processing, or OLAP
OLAP
OLAP is an approach to answer multi-dimensional analytical (MDA) queries swiftly in computing.OLAP is part of the broader category of business intelligence, which also encompasses relational databases, report writing and data mining.Typical applications of OLAP include business reporting for sales, marketing, management reporting, business process management (BPM), budgeting and forecasting, financial reporting and similar areas, with new applications emerging, such as agriculture.
Difference Between OLTP and OLAP
Difference Between OLAP and OLTP
OLTP and OLAP both are the online processing systems. OLTP is a transactional processing while OLAP is an analytical processing system. OLTP is a system that manages transaction-oriented applications on the internet for example, ATM. OLAP is an online system that reports to multidimensional analytical queries like financial reporting, forecasting, etc. The basic difference between OLTP and OLAP is that OLTP is an online database modifying system, whereas, OLAP is an online database query answering system.
key analytical operations
OLAP consists of three basic analytical operations: consolidation (roll-up), drill-down, and slicing and dicing. Consolidation involves the aggregation of data that can be accumulated and computed in one or more dimensions.
key analytical operations
Consolidation
Consolidation
Consolidation involves the aggregation of data that can be accumulated and computed in one or more dimensions. For example, all sales offices are rolled up to the sales department or sales division to anticipate sales trends.
Drill-down
Drill-down is a technique that allows users to navigate through the details. For instance, users can view the sales by individual products that make up a region's sales
Drill-down
Slicing and dicing
Slicing and dicing is a feature whereby users can take out (slicing) a specific set of data of the OLAP cube and view (dicing) the slices from different viewpoints. These viewpoints are sometimes called dimensions (such as looking at the same sales by salesperson, or by date, or by customer, or by product, or by region, etc.)
applications of OLAP
Typical applications of OLAP include business reporting for sales, marketing, management reporting, business process management (BPM),budgeting and forecasting, financial reporting and similar areas, with new applications coming up, such as agriculture. The term OLAP was created as a slight modification of the traditional database term OLTP (Online Transaction Processing).
types of OLAP systems
OLAP systems have been traditionally categorized using the following taxonomy
types of OLAP systems
MOLAP
MOLAP (multi-dimensional online analytical processing) is the classic form of OLAP and is sometimes referred to as just OLAP. MOLAP stores this data in an optimized multi-dimensional array storage, rather than in a relational database. Some MOLAP tools require the pre-computation and storage of derived data, such as consolidations – the operation known as processing. Such MOLAP tools generally utilize a pre-calculated data set referred to as a data cube. The data cube contains all the possible answers to a given range of questions. As a result, they have a very fast response to queries. On the other hand, updating can take a long time depending on the degree of pre-computation. Pre-computation can also lead to what is known as data explosion.
Multidimensional OLAP (MOLAP)
Advantages of MOLAP
- Fast query performance due to optimized storage, multidimensional indexing and caching.
- Smaller on-disk size of data compared to data stored in relational database due to compression techniques.
- Automated computation of higher level aggregates of the data.
- It is very compact for low dimension data sets.
- Array models provide natural indexing.
- Effective data extraction achieved through the pre-structuring of aggregated data.
Disadvantages of MOLAP
- Within some MOLAP systems the processing step (data load) can be quite lengthy, especially on large data volumes. This is usually remedied by doing only incremental processing, i.e., processing only the data which have changed (usually new data) instead of reprocessing the entire data set.
- Some MOLAP methodologies introduce data redundancy.
ROLAP
Relational OLAP (ROLAP)
ROLAP works directly with relational databases and does not require pre-computation. The base data and the dimension tables are stored as relational tables and new tables are created to hold the aggregated information. It depends on a specialized schema design. This methodology relies on manipulating the data stored in the relational database to give the appearance of traditional OLAP's slicing and dicing functionality. In essence, each action of slicing and dicing is equivalent to adding a "WHERE" clause in the SQL statement. ROLAP tools do not use pre-calculated data cubes but instead pose the query to the standard relational database and its tables in order to bring back the data required to answer the question. ROLAP tools feature the ability to ask any question because the methodology is not limited to the contents of a cube. ROLAP also has the ability to drill down to the lowest level of detail in the database.
Advantages of ROLAP
- ROLAP is considered to be more scalable in handling large data volumes, especially models with dimensions with very high cardinality (i.e., millions of members).
- With a variety of data loading tools available, and the ability to fine-tune the extract, transform, load (ETL) code to the particular data model, load times are generally much shorter than with the automated MOLAP loads.
- The data are stored in a standard relational database and can be accessed by any SQL reporting tool (the tool does not have to be an OLAP tool).
- ROLAP tools are better at handling non-aggregatable facts (e.g., textual descriptions). MOLAP tools tend to suffer from slow performance when querying these elements.
- By decoupling the data storage from the multi-dimensional model, it is possible to successfully model data that would not otherwise fit into a strict dimensional model.
- The ROLAP approach can leverage database authorization controls such as row-level security, whereby the query results are filtered depending on preset criteria applied, for example, to a given user or group of users (SQL WHERE clause).
Disadvantages of ROLAP
- There is a consensus in the industry that ROLAP tools have slower performance than MOLAP tools. However, see the discussion below about ROLAP performance.
- The loading of aggregate tables must be managed by custom ETL code. The ROLAP tools do not help with this task. This means additional development time and more code to support.
- When the step of creating aggregate tables is skipped, the query performance then suffers because the larger detailed tables must be queried. This can be partially remedied by adding additional aggregate tables, however it is still not practical to create aggregate tables for all combinations of dimensions/attributes.
- ROLAP relies on the general purpose database for querying and caching, and therefore several special techniques employed by MOLAP tools are not available (such as special hierarchical indexing). However, modern ROLAP tools take advantage of latest improvements in SQL language such as CUBE and ROLLUP operators, DB2 Cube Views, as well as other SQL OLAP extensions. These SQL improvements can mitigate the benefits of the MOLAP tools.
- Since ROLAP tools rely on SQL for all of the computations, they are not suitable when the model is heavy on calculations which don't translate well into SQL. Examples of such models include budgeting, allocations, financial reporting and other scenarios.
HOLAP
Hybrid OLAP (HOLAP)
The undesirable trade-off between additional ETL cost and slow query performance has ensured that most commercial OLAP tools now use a "Hybrid OLAP" (HOLAP) approach, which allows the model designer to decide which portion of the data will be stored in MOLAP and which portion in ROLAP.
There is no clear agreement across the industry as to what constitutes "Hybrid OLAP", except that a database will divide data between relational and specialized storage.[15] For example, for some vendors, a HOLAP database will use relational tables to hold the larger quantities of detailed data, and use specialized storage for at least some aspects of the smaller quantities of more-aggregate or less-detailed data. HOLAP addresses the shortcomings of MOLAP and ROLAP by combining the capabilities of both approaches. HOLAP tools can utilize both pre-calculated cubes and relational data sources.
Other types
Others
The following acronyms are also sometimes used, although they are not as widespread as the ones above:
- WOLAP – Web-based OLAP
- DOLAP – Desktop OLAP
- RTOLAP – Real-Time OLAP
- GOLAP – Graph OLAP[17][18]
- CaseOLAP – Context-aware Semantic OLAP, developed for biomedical applications. The CaseOLAP platform includes data preprocessing (e.g., downloading, extraction, and parsing text documents), indexing and searching with Elasticsearch, creating a functional document structure called Text-Cube, and quantifying user-defined phrase-category relationships using the core CaseOLAP algorithm.