Active Neural Networks to Detect Mentions of Changes to Medication Treatment in Social Media
My main interest is the quality of the annotations produced by NLP systems. My PhD thesis puts in evidence that, to date, no NLP system is able to produce automatically perfect annotations. Consequently, it is important to design NLP systems based on inference models dealing with uncertain information. During my first PostDoc I worked in close interaction with users from different domains. This was an opportunity to evaluate the usability of current NLP approaches according to the user’s point of view. It seems that there is a certain threshold beyond which users will regard the output of an NLP system as reliable, and that current systems have not yet reached that point. This is particularly true for systems which produce semantic information (e.g. Anaphora Resolution or semantic frames extraction). Their use can even be obtrusive if they present noisy and distracting information to the user. I have been recently working on the problem of structured prediction with graphical models and constrained conditional models. These Machine Learning techniques predict jointly values of several random variables along with their relations. In this expressive framework linguistic constraints are easily expressed and integrated in the inference model to remove likely but inadequate solutions. I’m currently applying these techniques on the task of geographical relation extraction from medical texts to help phylogeography studies.
Objective: We address a first step towards using social media data to supplement current efforts in monitoring population-level medication non-adherence: detecting changes to medication treatment. Medication treatment changes, like changes to dosage or to frequency of intake, that are not overseen by a physician are, by that, non-adherence to medication. Despite the consequences, including worsening health conditions or death, 50% of patients are estimated to not take medications as indicated. Current methods to identify non-adherence have major limitations. Direct observation may be intrusive or expensive, and indirect observation through patient surveys relies heavily on patients’ memory and candor. Using social media data in these studies may address these limitations.
Methods: We annotated 9,835 tweets mentioning medications and trained a convolutional neural network (CNN) to find mentions of medication treatment changes, regardless of whether the change was recommended by a physician. We used active and transfer learning from 12,972 reviews we annotated from WebMD to address the class imbalance of our Twitter corpus. To validate our CNN and explore future directions, we annotated 1,956 positive tweets as to whether they reflect non-adherence and categorized the reasons given.
Results: Our CNN achieved state-of-the-art performance with 0.50 F1-score. The manual analysis of positive tweets revealed that non-adherence is evident in a subset with nine categories of reasons for non-adherence.
Conclusion: We showed that social media users publicly discuss medication treatment changes and may explain their reasons including when it constitutes non-adherence. This approach may be useful to supplement current efforts in adherence monitoring.
KeywordsSocial Media, Pharmacovigilance, Medication Non-Adherence, Active Learning, Text Classification
Commenting is now closed.
To understand health and disease today, we need new thinking and novel science —the kind we create when multiple disciplines work together from the ground up. That is why this department has put forward a bold vision in population-health science: a single academic home for biostatistics, epidemiology and informatics.