Skip to content

How Document Analysis application will work

CIRSImg

Application Objective

Document Analysis is an application intended for retrieving clinical information from patient documents. By using this information we build CAC (Computer Assisted Coding), Auto Query Alert System, Patient Admit Criteria Forms Automation and lot of other health care applications.

Application Scope

Currently application scope is limited to well text formatted documents. Documents in other formats like PDF,text Images or OCR..etc not supported.

What type of clinical information available in patient documents

Diseases, Signs and Symptoms, Procedures, Findings, Laboratory Tests, Medications, Vital signs, Treatment plan, admit information, past medical history ..etc

Technical Stack and Knowledge resources used in Application

  • NLP Engine: [GATE](https://gate.ac.uk)[^1]
  • Java 8: [Java](http://www.oracle.com/technetwork/java/javase/overview/java8-2100321.html)
  • Health care domain dictionaries
  • [JAPES](https://gate.ac.uk) for Medical Coding
  • Web Services: For DRG Grouping

    • Document processing using GATE work flow

    Step 1: Embedding GATE Framework into Application

    A. Initiating GATE
    1
2
        System.setProperty("gate.home", ConnectionFactory.getGateHome());
    Gate.init();
    B.Load GATE Plug-ins
    1
2
3
4
         File pluginsHome = new File(Gate.getGateHome(), "plugins");
     Gate.getCreoleRegister().registerDirectories(new File(pluginsHome, "ANNIE").toURI().toURL());
     Gate.getCreoleRegister().registerDirectories(new File(pluginsHome, "ofai").toURI().toURL());
     Gate.getCreoleRegister().registerDirectories(new File(pluginsHome, "StringAnnotation").toURI().toURL());

    Step 3: Document Preprocessing

    some times document will get in different formats from same facility like, document may be in base64 format, or some times there is no proper line breaks conversion in Physician reports, to analysis documents properly we need specified format so we convert those documents in intended format 

    1
2
        // will get and preprocess all pending documents in 3,6,9 status
    {call GetDocFormatInDateDict()}

    Basically in above procedure we will decide document type by Document Type ID that is in 3,6,9.
    3,6 will denote that is physician document that need some line break replacement issues. and
    9 means source document is Base 64 format we will convert it into text format.

    Step 4: Retrieving Document by work type

    There are 4 types of analysis based on document work type namely

  • History and Physical Report

  • Operative Report
  • DIREP Report
  • Response Query Report
    • 1
2
3
4
5
6
7
        List.add("H");
    List.add("O");
    List.add("DIREP");
    List.add("RESQRY");
    while(String worktype:List){
    getHPMCDocument(worktype);
    }

    Query for getting pending documents by work type

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
        Select Top 100 DD.JobID,DD.SiteID,DD.Document,di.WorktypeId,di.Worktype,
    di.datedictcreated,di.AccountNo,v.PatientClass,DATEDIFF(MONTH,p.DOB,v.AdmitDate)/12 as 
    Age,
    DATEDIFF(DAY,p.DOB,v.AdmitDate) as AgeInDays,p.Sex,di.ReviewDay
    From Datedict DD
    inner join DocumentInfo DI on DD.JobId=DI.JobId
    inner join Visits v on v.AccountNumber=di.AccountNo
    inner join Patients p on p.PatientId=v.PatientID
    Where DD.Status=0 and di.ReviewDay is not null
    and (v.patientclass='I' and v.HospitalService not in ('13','16','30','40','71','94','96','97','VE','VB'))
    and v.admitdate is not NULL and v.admitdate<=GETDATE() and v.ploc_room!='' and Di.worktype in('RESQRY')

    Step 5: Building Document Corpus

    A: Create Corpus

    From step 4 we will get all pending documents that need to be analyze based on work type. The next step in the process is to build corpus for all these set of document using GATE gate.corpora.CorpusImpl class

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
        try {
            corp = (Corpus) Factory.createResource("gate.corpora.CorpusImpl");
            do {
                docCount++; int jobId = 0;
                try {
                    jobId = rs.getInt(1);
                    docList += jobId + ",";
                    fsiteId = rs.getInt(2);
                    String str = rs.getString(3);
                    FeatureMap features = Factory.newFeatureMap();
                    features.put("jobId", jobId);
                    // Other Required features added here ...
                    gate.Document document =  
                       (gate.Document) Factory.createResource("gate.corpora.DocumentImpl");
                    int wtid = Integer.parseInt(rs.getString(4));
                    if (wtid >= 200) {
                        features.put("mimeType", "text/html");
                    }
                    document.setFeatures(features);
                    document.setContent(new DocumentContentImpl(str));
                    document.setName(jobId + "");
                    corp.add(document);
                    document = null;
                    documentIds += jobId + ",";
                } catch (Exception e1) {
                    try {
                        writeLog(e1, "\n\n" + "problem with the document" + jobId);
                        getUpdateRunningDocCount();
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }

            } while (docCount <= 100 && rs.next());//while
            docList = docList.substring(0, docList.length() - 1);

        } catch (Exception e) {
            writeLog(e, "Failed while building corpus");
        } finally {
            return docList;
        }
    B: Set Running status for documents

    After building corpus, we are ready to process documents, to make aware of these documents are running to front end user we will update status with 2, that means all status 2 documents are currently under running process.

    1
2
3
4
5
6
7
8
         Connection con = ConnectionFactory.getInstance().getConnection();
            Statement st = con.createStatement();
            st.executeUpdate
            ("update DateDict set status=2 where jobid 
             in(" + docList + ") and status=0");
            st.close();
            con.close();
            System.gc();
    C: Delete Modified Cached List file Gaz bins

    In this we will delete cached list files is any are modified and will recreate Gaz bins again using update list files from system. 

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
    for (File file : fList) {
            if (file.isFile()) {
             if (file.getName().endsWith(".lst")){
                DeleteGazbinFile.deleteGazbin1(file.getParent(),
                file.getAbsolutePath(),file.getName(),file);
             }
            } else if (file.isDirectory()) {
                resultList.addAll(deleteGazbin(file.getAbsolutePath()));
            }
        }

    Step 6: Document Processing with GATE default Resources & User defined list files (Dictionaries)

    A: Creation of GATE Controller

    Note

    Controllers are used to create GATE applications. A Controller handles a set of Processing Resources and can execute them following a particular strategy.GATE provides a series of serial controllers (i.e. controllers that run their PRs in sequence):

    GATE default coder for Creating an ANNIE application and running it over a corpus check GATE Site

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
    Creating an ANNIE application and running it over a corpus
// load the ANNIE plugin 
Gate.getCreoleRegister().registerDirectories(new File( 
 Gate.getPluginsHome(), "ANNIE").toURI().toURL()); 

// create a serial analyser controller to run ANNIE with 
SerialAnalyserController annieController = 
 (SerialAnalyserController) Factory.createResource( 
    "gate.creole.SerialAnalyserController", 
    Factory.newFeatureMap(), 
     Factory.newFeatureMap(), "ANNIE"); 

// load each PR as defined in ANNIEConstants 
for(int i = 0; i < ANNIEConstants.PR_NAMES.length; i++) { 
  // use default parameters 
  FeatureMap params = Factory.newFeatureMap(); 
  ProcessingResource pr = (ProcessingResource) 
      Factory.createResource(ANNIEConstants.PR_NAMES[i], 
                             params); 
  // add the PR to the pipeline controller 
  annieController.add(pr); 
} // for each ANNIE PR 

// Tell ANNIE’s controller about the corpus you want to run on 
Corpus corpus = ...; 
annieController.setCorpus(corpus); 
// Run ANNIE 
annieController.execute();

    Document Analysis used code for controller creation

    1
2
3
    SerialAnalyserController annieController = (SerialAnalyserController) 
                    Factory.createResource("gate.creole.SerialAnalyserController",
                    Factory.newFeatureMap(), Factory.newFeatureMap(), "ANNIE_" + Gate.genSym());
    B: Add processing resources to controller and running corpus with resources

    In this section we will add GATE default processing resources to controller and will process corpus with this resources. 

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
        FeatureMap params = Factory.newFeatureMap();

    // Adding Annotation Reset PR
    ProcessingResource deletePR = 
    (ProcessingResource) Factory.
    createResource("gate.creole.annotdelete.AnnotationDeletePR", params);
    annieController.add(deletePR);

    // Adding Token PR
    ProcessingResource tokenPR = 
    (ProcessingResource) Factory.
    createResource("gate.creole.tokeniser.DefaultTokeniser", params);
    annieController.add(tokenPR);

    // Adding Default Gazetteer PR
    ProcessingResource gazetteerPR = 
    (ProcessingResource) Factory.
    createResource("gate.creole.gazetteer.DefaultGazetteer", params);
    annieController.add(gazetteerPR);

    // Executing Corpus with Above resources
    annieController.setCorpus(corp);
    annieController.execute();

    // Removing Default Annotation reset PR
    annieController.remove(deletePR);
    Factory.deleteResource(deletePR);

    // Removing Token PR
    annieController.remove(tokenPR);
    Factory.deleteResource(tokenPR);

    // Removing Default Gazetteer PR 
    annieController.remove(gazetteerPR);
    Factory.deleteResource(gazetteerPR);

    // Making all Resource variable to NULL
    deletePR = null;tokenPR = null;gazetteerPR = null;

    // Running User Defined Dictionaries using Extended gazetteer 2  
    runGazetteer();
    runExtendedGazetteer();

    // Sentence Splitter Resource
    ProcessingResource sentencePR = 
    (ProcessingResource) Factory.
    createResource("gate.creole.splitter.SentenceSplitter", params);
    annieController.add(sentencePR);

    //POS Resource
    ProcessingResource posPR = 
    (ProcessingResource) Factory.createResource("gate.creole.POSTagger", params);
    annieController.add(posPR);

    // ANNIE TRANSDUCER Resources
    ProcessingResource anniePR = 
    (ProcessingResource) Factory.createResource("gate.creole.ANNIETransducer", params);
    annieController.add(anniePR);

    // Orthomatcher Resources
    ProcessingResource orthoPR = 
    (ProcessingResource) Factory
    .createResource("gate.creole.orthomatcher.OrthoMatcher", params);
     annieController.add(orthoPR);

    // Running Corpus with added resources
     annieController.execute();

    // Removing Resources from Controller
     annieController.remove(sentencePR);
     annieController.remove(posPR);
     annieController.remove(anniePR);
     annieController.remove(orthoPR);

    // Deleting resources from Factory
     Factory.deleteResource(sentencePR);
     Factory.deleteResource(posPR);
     Factory.deleteResource(anniePR);
     Factory.deleteResource(orthoPR);

    // making resource variable to NULL
    sentencePR = null;posPR = null;anniePR = null;orthoPR = null;

    // Running Java Garbage COllection        
    System.gc();
    C: User Defined Gazetteers (Clinical Dictionary)

    In above section at line 41,42 we call to external Gazetteer processing methods. In this section we will show how and what are the user defined list and dictionary files we will add to the system

    Default dictionary files irrelevant to activated modules

    Dictionary File Path Dictionary File Name Purpose
    MedResource/Adjectives Adjectives.def Describe the purpose of the dictionary
    MedResource/Adjectives Condition.def Describe the purpose of the dictionary
    MedResource/BodyPart BodyLocation.def Describe the purpose of the dictionary
    MedResource/BodyPart BodyPart.def Describe the purpose of the dictionary
    MedResource/BodyPart BodySpace.def Describe the purpose of the dictionary
    MedResource/BodyPart BodySubstance.def Describe the purpose of the dictionary
    MedResource/BodyPart BodySystem.def Describe the purpose of the dictionary
    MedResource/General SectionHeadings.def Describe the purpose of the dictionary
    MedResource/General PdocHeadings.def Describe the purpose of the dictionary
    MedResource/LabData LaboratoryProcedure2.def Describe the purpose of the dictionary
    MedResource/Medications Medication.def Describe the purpose of the dictionary
    MedResource/Procedure DiagnosticProcedure.def Describe the purpose of the dictionary
    MedResource/Procedure LaboratoryProcedure.def Describe the purpose of the dictionary
    MedResource/Procedure PreventiveProcedure.def Describe the purpose of the dictionary
    MedResource/Subheadings Subheadings.def Describe the purpose of the dictionary
    MedResource/DischargeSummary DischargeCondition.def Describe the purpose of the dictionary
    DiseaseJape list.def Describe the purpose of the dictionary
    DiseaseJape prenegation.def Describe the purpose of the dictionary
    DiseaseJape site.def Describe the purpose of the dictionary
    DiseaseJape dlist1.def Describe the purpose of the dictionary
    DiseaseJape dlist2.def Describe the purpose of the dictionary
    DiseaseJape dlist3.def Describe the purpose of the dictionary
    DiseaseJape dlist5.def Describe the purpose of the dictionary
    DiseaseJape/stage stage.def Describe the purpose of the dictionary
    DiseaseJape ESignReport.def Describe the purpose of the dictionary
    IDiseaseJape IDisorder.def Describe the purpose of the dictionary
    IDiseaseJape ISite.def Describe the purpose of the dictionary
    DiseaseJape Habits.def Describe the purpose of the dictionary
    Cancer cancer.def Describe the purpose of the dictionary
    Radiology RadiologyHeadings.def Describe the purpose of the dictionary
    DiseaseJape/CoreMeasures CoreMeasure.def Describe the purpose of the dictionary
    JAPE/Laboratory ResLab.def Describe the purpose of the dictionary

    Procedure Related Dictionary files

    Dictionary File Path Dictionary File Name Purpose
    MedResource/ICD PCS.def Describe the purpose of the dictionary
    MedResource/ICD/LST PCSComb.def Describe the purpose of the dictionary
    MedResource/ICD/PCSGeneral PCSGeneral.def Describe the purpose of the dictionary
    MedResource/ICD PCS_BP.def Describe the purpose of the dictionary
    MedResource/ICD/Operations/OPList DiscreteOperations.def Describe the purpose of the dictionary
    MedResource/ICD/Operations/OPList DiscreteSites.def Describe the purpose of the dictionary
    MedResource/ICD/Operations/OPList PTADb.def Describe the purpose of the dictionary
    MedResource/ICD/Operations/OPList PortCath.def Describe the purpose of the dictionary

    E&M Code Related Dictionary Files

    Dictionary File Path Dictionary File Name Purpose
    MedResource/EMCode/History EMCodeHeaders.def Describe the purpose of the dictionary
    MedResource/EMCode/History HPIElements.def Describe the purpose of the dictionary
    MedResource/EMCode/History HPISignorSymptom.def Describe the purpose of the dictionary
    MedResource/EMCode/Examination ExamElements.def Describe the purpose of the dictionary
    MedResource/EMCode/MDMElements/DTOptions DTOptions.def Describe the purpose of the dictionary
    MedResource/EMCode/MDMElements/AmountAndComplexity ACDataReviewed.def Describe the purpose of the dictionary
    MedResource/EMCode/MDMElements/RiskLevel MdmRiskElements.def Describe the purpose of the dictionary

    SNOMEDCT Related Dictionary Files

    Dictionary File Path Dictionary File Name Purpose
    DiseaseJape/SnomedCT Snomedct.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT Snomedct1.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT DiseaseOrSyndrome.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT DiseaseOrSyndrome1.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT DiseaseOrSyndrome2.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT TherapeuticPreventiveProcedure.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT TherapeuticPreventiveProcedure1.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT Bacterium.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT Bacterium2.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT Finding.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT Finding1.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT Fungus.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT InjuryOrPoisoning.def Describe the purpose of the dictionary
    DiseaseJape/SnomedCT NeoplasticProcess.def Describe the purpose of the dictionary

    Query Alert Related Dictionary Files

    Dictionary File Path Dictionary File Name Purpose
    DiseaseJape queryword.def Describe the purpose of the dictionary

    Interqual Related Dictionary Files

    Dictionary File Path Dictionary File Name Purpose
    DiseaseJape/IQAnalysis IQData.def Describe the purpose of the dictionary

    Milliman Related Dictionary Files

    Dictionary File Path Dictionary File Name Purpose
    DiseaseJape/MillimanAnalysis MillimanData.def Describe the purpose of the dictionary

    Compliance Audit Related Dictionary Files

    Dictionary File Path Dictionary File Name Purpose
    ComplianceAudit Compliance.def Describe the purpose of the dictionary

    Processing Corpus with above Dictionary Files

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
         ArrayList<String> defList = getRequiredDefs();
     for (int i = 0; i < defList.size(); i++) {
      try {
            System.gc();
            FeatureMap gparams = Factory.newFeatureMap();
            gparams.put("configFileURL", (new File(japeDefPath + defList.get(i))).toURL());
            gparams.put("caseSensitive", "false");
            gparams.put("gazetteerFeatureSeparator", "~");
            ProcessingResource spr = (ProcessingResource) 
            Factory.createResource("com.jpetrak.gate.stringannotation.
              extendedgazetteer2.ExtendedGazetteer2", gparams);
            annieController.add(spr);
            annieController.execute();
            annieController.remove(spr);
            Factory.deleteResource(spr);
            spr = null;
            System.gc();
                }

    **For Some other Module Dictionary files processing **

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
        String files[] = {"DiseaseJape/vseriesdef.def", "DiseaseJape/Wseries.def", 
    "DiseaseJape/Yseries.def", "Fracture/Fractures.def", "MedResource/ICD/PCSGazbin.def"};
    for (int i = 0; i < files.length; i++) {
         try {
              System.gc();
              FeatureMap gparams = Factory.newFeatureMap();
              gparams.put("configFileURL", (new File(japeDefPath + files[i])).toURL());
              gparams.put("caseSensitive", "false");
              gparams.put("gazetteerFeatureSeparator", "~");
              ProcessingResource spr = (ProcessingResource) Factory.createResource("com.jpetrak.gate.stringannotation.extendedgazetteer2.ExtendedGazetteer2", gparams);
              annieController.add(spr);
              annieController.execute();
              annieController.remove(spr);
              Factory.deleteResource(spr);
              spr = null;
              System.gc();
                } catch (Exception def) {
                    writeLog(def, def.toString());
                }
    D: User Defined JAPE2 for Clinical Information Pattern Recognition

    JAPE is a Java Annotation Patterns Engine. JAPE provides finite state transduction over annotations based on regular expressions. JAPE is a version of CPSL – Common Pattern Specification Language.

    As part of this we use almost 2453 unique JAPE files to retrieve clinical information from documents based on different modules requirement.

    ** Code for how JAPE are added to Resource and process over corpus**

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
        try {
            // We are using different controller for skipping rule
             // if rule run more than 3 minutes
            RealtimeCorpusController rtController = (RealtimeCorpusController)
            Factory.createResource("gate.creole.RealtimeCorpusController", 
            Factory.newFeatureMap(), Factory.newFeatureMap(), "ANNIE_" + Gate.genSym());

            rtController.setTimeout(30000L);
            rtController.setCorpus(corp);

            for (int i = 0; i < japeList.size(); i++) {

                FeatureMap params = Factory.newFeatureMap();
                params.put("grammarURL", (new File(japeDefPath + japeList.get(i))).toURL());
                ProcessingResource prs = (ProcessingResource) 
                Factory.createResource("gate.creole.Transducer", params);
                rtController.add(prs);
          // Why we are adding another test JAPE I don't know                
                if (testjape.contains(japeDefPath + japeList.get(i))) {
                    FeatureMap params2 = Factory.newFeatureMap();
                    String testfile = (japeDefPath + japeList.get(i))
                    .substring(0, (japeDefPath + japeList.get(i)).length() - 5);
                    params2.put("grammarURL", 
                    (new File(testfile + "_Test.jape")).toURI().toURL());

                    ProcessingResource prs2 = (ProcessingResource) 
                    Factory.createResource("gate.creole.Transducer", params2);

                    rtController.add(prs2);
                    rtController.execute();

                    rtController.remove(prs);
                    rtController.remove(prs2);
                    Factory.deleteResource(prs);
                    Factory.deleteResource(prs2);
                    prs = null;
                    prs2 = null;

                    System.gc();

                } else {
                    rtController.execute();
                    rtController.remove(prs);
                    Factory.deleteResource(prs);
                    prs = null;
                    System.gc();
                    timeDuration("Time for " + japeList.get(i));
                }

            }//for
            Factory.deleteResource(rtController);
            rtController = null;
        }

    Summary

    This will complete analysis of document using GATE and user defined JAPES, from here application will work based on Module specific requirements Ex: IF ICD code is target module then it will look for ICD code based on disease combination annotation features.

    Reading Clinical Information from Corpus (Old Analysis)

    A: Delete previous information from tables by Job ID and Database

    In this method application will delete all information from tables on current analysis job id

    Procedure Name:DeletedAllTableDataByJobId

    Tables: NA (All Annotation related tables)

    B: Reading Medications Information

    All Medication related information from patient documents will read here

    currently not using

    This is old annotation currently no information is inserting from this set please check procedure This

    Input Annotation Set:PatientDetails

    Procedure Name: InsertMedInfo

    Table: Med

    C: Reading Patient Demographic Information

    In this application will read patient age, Country, gender information from documents

    Input Annotation Set: PatientDetails

    Procedure/Query Name: insert into PatientInfo (docId,siteId,Worktype,WorktypeId,AnnotId,Age,value1,Country,Gender,startId,endId) values(?,?,?,?,?,?,?,?,?,?,?)

    Table: PatientInfo

    D: Reading Review of systems (ROS) information

    In this application will read Review of systems (ROS) information from documents

    Input Annotation Set: Review_Of_Systems1, Review_Of_Systems

    Procedure/Query Name:InsertROSInfo (......)

    Table: Reviewofsystems

    E: Reading Review of systems NEG (ROS) information

    Currently it is trying to insert ROS sub headings in to table, but inserting all null values

    Null Insertion

    Currently inserting null sub headings into table

    Input Annotation Set: ROSNeg

    Procedure/Query Name:insert into ROSNeg (docId,Subheading) values(?,?)

    Table: Rosneg

    E: Reading Physical Examination (PExam) Information

    reading patient physician exam information

    Input Annotation Set: PhysicalExamination

    Procedure/Query Name: ----

    Table: Pexamcontent, PExam

    F: Reading Vital Signs Information

    reading patient vital information

    Input Annotation Set: PhysicalExamination2

    Procedure/Query Name: ----

    Table: Vsigns

    G: Reading Physician treatment plan Information

    reading patient treatment plan information

    Input Annotation Set: Plan03

    Procedure/Query Name: InsertPlanRecordInfo (-----)

    Table: Plan4, planrecord

    H: Reading Patient Admit order Information

    reading patient admit orders like , 2 day in patient, admitted as in patient etc

    Input Annotation Set: Admitdata2, Admitdata1, ESignFinal

    Procedure/Query Name: InsertAdmitOrders (-----)

    Table: Admitorders

    I: Reading Patient Laboratory data Information

    reading patient laboratory test related information, value is not separated

    Input Annotation Set: LaboratoryData

    Procedure/Query Name: InsertLabContentInfo (-----)

    Table: LabContent

    reading patient Procedure related information regarding ICD 9

    Old Version and specific to Work type O

    Reading procedure related information regarding ICD 9

    Input Annotation Set: Procedure1

    Procedure/Query Name: GetCPTCodeNew (-----)

    Table: Procedure

    reading patient DIagnosis related information regarding ICD 9

    Old Version and specific to Work type O

    Reading Diagnosis related information regarding ICD 9, but not inserting

    Input Annotation Set: Diagnosis2

    Procedure/Query Name: GetICDCode (-----), no procedure in DB with This name there is another procedure GetICDCode1

    Table: Disorder

    reading patient Discharge DIagnosis related information

    old version and specific to Work type D

    Reading Discharge Diagnosis related information

    Input Annotation Set: Discharge Diagnosis

    Procedure/Query Name: GetICDCode (-----), no procedure in DB with This name there is another procedure GetICDCode1

    Table: Disorder

    reading patient Discharge Condition, Activity Level, diet information

    specific to Work type D

    Reading Discharge Diagnosis related information

    Input Annotation Set: DischargeCondition, Activity Level, DischargeDiet

    Procedure/Query Name: InsertDischargeCOnditionInfo

    Table: DischargeCondition

    L: Non Using Annotation SETS

    Following are not running in application currently

    Not Running currently

    Following Annotation sets are currently not Processing

    Input Annotation Set: Allergies, Complaint, FHistory, Habits5, Impression, Past_Medical_History, SurgicalHistory, HPILocation , Severity, Timing, Present_Illness

    Procedure/Query Name: NA

    Table: NA

    Reading Clinical Information from Corpus (New Analysis, Annotation Processor)

    Analysis of corpus from AnootationProcessor Java files

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
        if (runOtherAnnotations) {
     AllDocumentAnatations adAnnotations = new AllDocumentAnatations(corp);
     adAnnotations.analyzeCorpus();
     adAnnotations = null;
     }
    if (runEmcode) {
     EmcodeAnalysis emAnalysis = new EmcodeAnalysis(corp);
     emAnalysis.analyzeCorpus();
     emAnalysis = null;
    }
   if (runDBInsert) {
     AnnotationProcessor historyAnalyzer = new AnnotationProcessor(corp);
     historyAnalyzer.analyzeCorpus();
     }
   if (runDataStore) {
      setDatastore(workType, docList);
        }

    Process of all modules and corresponding annotations

    Compliance Audits

    Method Name: getComplianceAudit

    Input Annotation Set: ComplianceCE,ComplianceDST1,ComplianceTT1,ComplianceFinal

    Procedure Name: GetEncphKeywordsList

    Table Name: EncphkeyAnnotations

    Work Type: All

    Description: To Insert Compliance audit related keywords

    SNOMEDCT Code

    Method Name: getSnomedCtKeywords_TTY

    Input Annotation Set: SnomedCT4, SnomedCT5, SnomedCT6, SnomedCT7, SnomedCT9, SnomedCT10, SnomedDiff

    Procedure Name: GetSnomedCtwordsList

    Table Name: SnomedTerms

    Work Type: All

    Description: To Insert SNOMEDCT Keywords with code

    All Clinical Annotations

    Method Name: getSignOrSymptomsInfo

    Input Annotation Set: Alldis

    Procedure Name: Direct Query

    Table Name: signorsymptom

    Work Type: All

    Description: To Insert all document clinical words in to table

    Radiology Report Annotations

    Method Name: getRadSymptomsInfo

    Input Annotation Set: RadDis

    Procedure Name: Direct Query

    Table Name: Radsymptoms

    Work Type: All (Radiology)

    Description: To Insert radiology clinical annotations

    Medical Subject Headings

    Method Name: getSectionHeaders

    Input Annotation Set: SectionHeader

    Procedure Name: Direct Query

    Table Name: SectionHeaders

    Work Type: All

    Description: To Insert all section headings from documents

    Vital signs

    Method Name: getVitalSigns

    Input Annotation Set: VitalsTest, VitalSigns, VitalSigns1

    Procedure Name: InsertVitalSigns

    Table Name: SectionHeaders

    Work Type: All

    Description: To Insert all section headings from documents

    **Following are the complete module list implemented in Application **

    getSymptomAnnotations(); insertMedicationDataIntoDB(); // Implemented to get Medications from Document to table insertHomeMedicationDataIntoDB(); insertImmunizationDataIntoDB(); insertAllergiesDataIntoDB(); insertHabitsDataIntoDB(); insertLabDataIntoDB(); // Implemented to get Labs from Document to table ICD10PregnancyInfo(); ICD10DiabetesInfo(poa); ICD10ComplaintInfo(); ICD10PresentIllnessInfo(); ICD10PastMedicalHistoryInfo(); ICD10AllergiesDiseaseInfo(); ICD10SampleHabitsInfo(); burnsInfo(); DrugsInfo(); ICD10BirthWeightInfo(); if (neoNatal == 1) { getNewbornInfo(poa); } ICD10ProcedureSymptomsKeywords(); ICD10ImpressionFromPMH(docNo, poa); ICD10BMI(); ICD10CheckObesity(); if (isPreg) { OBGMaternalDx(); timeDuration("OBGMaternalDx"); OBGDeliveryCount(); timeDuration("OBGDeliveryCount"); OBGHabitsInfo(); timeDuration("OBGHabitsInfo"); } DxDrugEffects(); fetchFxTerms2(); fetchFxTerms3(); DXKeywordInfo(); InsertLongTermDrugDiseases(accountNo, docNo, poa); ICD10DxComboChange(accountNo, docNo, poa); ICD10HtnAnalyser(accountNo); ICD10CHFAnalyser(accountNo); insertICD10(docNo, accountNo, worktype, poa); getWcodes(docNo, accountNo, worktype); getXcodes(docNo, accountNo, worktype); getVcodes(docNo, accountNo, worktype, poa); getYCodes(docNo, accountNo, worktype, poa); Icd10ExcludeCodes(accountNo); ICD10Codes(docNo, poa);

    //Procedure Coding Methods Here

    getProcedureDate(); findDirectPCS(); getAbsoluteAVSDBodyPart(); getAbsoluteBodyPart(); getAbsolutePTA(); getAbsoluteTRMB(); getHematomaOp(); getDiverticulectomy(); getBiopsyOperation(); getSkinFlapsOp(); getAmputation(); getEctomies(); getEctomyInspection(); getInsertionPortCath(); getAbsoluteEndarterectomy(); getEndarterectomyPatchAngioplasty(); getAbsoluteAtherectomy(); getAbsoluteFulguration(); getAbsoluteFractureRd2(); getAbsoluteOperation(); getPCS(); getKeyFeatures(); insertPCSInfo();

    ICD10POAUpdateByExemption(); getDocBodyParts(); getInterqualValue();

    E & M Code Analysis

    For Evaluation and Management we have separate analysis methods Please find following code

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
        if (runEmcode) {
    EmcodeAnalysis emAnalysis = new EmcodeAnalysis(corp);
    emAnalysis.analyzeCorpus();
    emAnalysis = null;
                            }
    try {
       document = (gate.Document) corpusIterator.next();
       docContent = document.getContent().toString();
       docNo = Integer.parseInt(document.getFeatures().get("jobId").toString());
       docAnnotationSet = document.getAnnotations();
       accountNo = (String) document.getFeatures().get("accountNo");
       patientClass = (String) document.getFeatures().get("patientClass");
       worktype = (String) document.getFeatures().get("worktype");
       getEMCodeHPIElements();
       getEMCodeROSElements();
       getEMCodeROSElements1();
       //getEMCodeROSElements2();
       getEMCodePFSH();
       getEMCodePExamBodyAreas();
       getEMCodePExamOrganSystems();
       getEMCodePExamEyeElements();
       getEMCodePExamENTMElements();
       getEMCodeVSigns();
       getEMCodeVSigns1();
       getEMCodeDTOptions();
       if (worktype.equalsIgnoreCase("ERR")) {
                  getErrEstProblems();
       }
       getEMCodeCDReviewed();
       getEMCodeRiskOfComplications();
       getPlanRecordInfo();
       String physicianName = getPhysicianName();
       getEMCode(physicianName, worktype, patientClass);
       document = null;
                docContent = null;
                worktype = null;
                docAnnotationSet = null;
            }

    Saving Corpus Data Store (Analysis status)

    By this method Application will store Corpus analysis state in file system

    1
2
3
4
5
6
        SerialDataStore sds = (SerialDataStore) 
    Factory.createDataStore("gate.persist.SerialDataStore", "file:///" + DS_DIR);
    sds.open();
    Corpus persistCorp = (Corpus) sds.adopt(corp);
    sds.sync(persistCorp);
    sds.close();

    Clearing All

    In this application will clear all data documents, corpus from memory

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
        for (Iterator corpusIterator = corp.iterator(); corpusIterator.hasNext();) {
    gate.Document d = (gate.Document) corpusIterator.next();
    corp.remove(d);
    Factory.deleteResource(d);
            }
    corp.clear();
    Factory.deleteResource(corp);
    annieController.cleanup();
    Factory.deleteResource(annieController);
    corp = null;
    annieController = null;

    Conclusion

    Conclusion

    The Primary intention for writing this application is to facilitate computer assisted medical coding feature to end user.But in the development process there is a lot of other clinical information has been analyzed and retrieved like Compliance Audits, Interqual, Milliman, Query Alerts, Admit orders and E & M Code.

    Following is the brief work flow of the application

    1. End user, client will define and provide requirements for data to be retrieved from clinical documents and will provide sample set of documents

    2. Developer will look into the context of requirement from lot of pre existing documents

    3. Next will gather required clinical information as List or Dictionary files

    4. Using GATE framework, it will generate primary annotation set (NER)

    5. Above that developer will write GRAMMAR (JAPE) rules according to requirement need

    6. Test on new documents, and continues trail and errors on new data and will finalize the new component and will integrate in existing system.

    Thank You

    Document By: Krishna Reddy 04/06/2018

    1. GATE stands for General Architecture for Text Engineering ↩

    2. JAPE stands for Java Annotation Pattern Engine ↩

    3. you can view the project for real JAPE names used ↩