Sentiment Analytics
Sentiment analysis involves indexing social media comments and using algorithms to determine whether they are positive or negative. By combining NLP (Natural Language Processing) and ML techniques, a sentiment analysis system for text analysis reads different texts from a variety of sources. It then stacks them against a sentiment library (a repository of adjectives that have been assigned a sentiment score based on usage and feedback from readers. It then assigns weighted sentiment scores to the entities, topics, themes, and categories within a sentence or phrase.
Sentiment analysis helps data analysts within large enterprises gauge public opinion, conduct nuanced market research, monitor brand and product reputation, and understand customer experiences. Data analytics companies often integrate third-party sentiment analysis APIs into their customer experience management, social media monitoring, or workforce analytics platform to deliver useful insights to their customers. Contextual mining of text, which identifies and extracts subjective information in the source material, helps a business to understand the social sentiment of their brand, product, or service while monitoring online conversations and enables better decision-making.
Challenges
Analyzing social media streams is restricted to sentiment analysis and count-based metrics that barely scratch the surface and leave out high-value insights. Sentiment analysis has problems predicting outcomes of dynamic events. It is highly challenging to train a successful model for conducting sentiment analysis on tweet streams for a dynamic event such as an election. Among the key challenges are changes in conversation topics and the people social media users post about.
Coreference resolution is the problem of identifying whether a phrase is a noun, pronoun, etc. For example, in the sentence “We watched the match and went to the restaurant; it was awful,” ‘it’ doesn’t clearly convey whether the match or the restaurant was ‘awful.’ However, coreference resolution may be useful for the topic/aspect based sentiment analysis and may improve the accuracy of opinion mining. To add to this, handling abbreviations, inaccurate use of uppercase and lowercase, poor spelling, punctuation, and grammar are some more areas that give rise to complications.
The time a review is done is important for sentiment analysis. A reviewer may think that a piece of software is good in 2010, but now he may have a negative opinion in 2020 due to newer versions or competitors. Because of this reality, assessing opinions that are time-sensitive can be problematic in sentiment analysis. However, changing opinions can also help observe if certain products improved over time and why.
Recognizing named entities that require world knowledge can affect sentiment analysis. For example, “Casablanca and a lunch comprising rice and fish: a good Sunday.” A sentiment analysis system without world knowledge classifies the above sentence as positive due to the word ‘good.’ Still, it is an objective sentence because Casablanca is the movie’s name. Also, the accuracy of sentiment classification can be influenced by the domain of the items to which it is applied. There are many words whose meaning changes from domain to domain, and many times text contains different words with the same meaning. So such words should be identified and grouped together for accurate classification. It is a difficult task to identify these words, as people often use different words to describe the same feature. For example, both ‘voice’ and ‘sound’ refer to the same feature in phone reviews.
Also, sarcasm can confuse sentiment analysis systems. Similarly, negation in traditional text classification small differences between two pieces of text don’t affect the meaning too much. In sentiment analysis, however, ‘the food was great’ is very different from ‘the food was not great,’ and negation handling is a difficult task in sentiment analysis as it reverses the polarity. Sarcasm and implicit sentences also express negation but don’t contain any negative words.
Solution Approach
A large healthcare provider catering to patients in Philadelphia, Pennsylvania, and New Jersey wanted to improve their patient experience by analyzing patient sentiment extracted from text comments in their patient feedback forms.
xpressso.ai platform provides out-of-the-box development frameworks. The project was started with the relevant environments, which were then created automatically. Development images configured based on pre-defined templates were installed on-premises or in a development VM within the xpresso.ai infrastructure. This enabled authentication using LDAP, seamless project setup using Bitbucket, Jenkins, and Docker (ensuring build and deployment without software compatibility issues).
The platform made available by xpresso.ai leverages the latest ML and DL tools while preparing models. It includes Pachyderm-based data versioning, Kubernetes, Kubeflow, and Spark-based ML and DL. It also includes an Istio-based service mesh-enabled microservice architecture, and ELK-based monitoring capability, contributing to a reduction in latency time.
xpresso.ai’s MLOps platform allows establishing high-end Alluxio and Presto-based efficient data connectivity and collecting data from diverse sources. The context from the patient feedback forms was captured by a word embedding layer (word2vec) and added as exploratory variables by using xpresso.ai libraries.
By using xpresso.ai data versioning and xpresso.ai connectivity libraries, data versions were easily controlled, stored into xpresso Data Model (XDM)-enabled data store. This enabled easy retrieval and storage of datasets/ files into internal XDM. The attributes obtained were used for categorization (employing Pachyderm-based data versioning) and then performing univariate, bi-variate, and Bag of Words analysis — for both structured and unstructured datasets through xpresso Exploratory Data Analysis (Data and Statistical Analysis). Different datasets and their different versions were easily controlled and stored into an xpresso Data Model (XDM)-enabled data store that enabled easy retrieval and storage of datasets/ files into internal XDM. This was achieved using two excellent features of xpresso.ai:
- Data Connectivity Marketplace libraries
- Data Versioning
We fed the output into the LSTM model to classify texts into desired categories.
The solution provided an accurate assessment of patient opinions about the hospital’s performance and helped our client to identify key pain points for patients through various stages of their healthcare journey. The client is witnessing a considerable increase in the positive sentiments and an increase in the performance ratings, which is translating into increased revenue.
How xpresso.ai can help Finance Organizations transform their journey to cognitive AI solutions
xpresso.ai is an AI/ML Application Lifecycle Management Platform.
xpresso.ai enables complete lifecycle management of AI/ML solutions, addressing the AI transformation journey of enterprises on any cloud platform of choice. xpresso.ai offers functionality essential for building AI/ML solutions – primarily enabling data scientists to rapidly build predictive and prescriptive models. The platform provides a user-friendly interface to develop, deploy, and manage AI/ML solutions at scale. In addition, xpresso.ai supports the incorporation of these solutions into business processes, surrounding infrastructure, products and applications.
Key benefits of xpresso.ai include:
- Empowers data scientists to transform AI/ML research into solutions
- Improves the productivity of data scientists by enabling them to focus on the business problem, developing algorithms and rapid experimentation of models
- Addresses the shortage of skilled data science resources with automated workflows, toolkits and frameworks
- Manages AI transformation journey costs without any wastage of R&D efforts
- Provides an enterprise-ready and secure environment for complete lifecycle management of AI/ML applications
- Enables at-scale deployment of enterprise AI/ML applications on-premise, cloud (AWS, GCP, Azure), or hybrid environments
Additional details on xpresso.ai can be found at: https://stg.xpresso.ai. We can schedule a demo of the platform for anyone interested in learning more.