a#nSc@sdZddlmZddlmZddlZddlZddlZddlZddlZyddl m Z m Z Wne k rdZ nXdZ ddlmZmZdd lmZdd lmZdd lmZejad Zdd ZdZddZddddZdddddddddZdddddddddZddddddddZ dZ!ddZ"dddddddddZ#ddZ$dddddZ%dddZ&dZ'ddZ(dZ)ddZ*d Z+e,d!kre(Z-e+e-Z.e*d"dgZ/e-j0re-j1re.dZ2ed#ee-j1e2Z3e3j4d$d%d&d'e-j0Z5xe5D]ZZ6e6dZ6y e/j7ej8e9e6Wqddl:Z:e:j;ed(qXqWn3ed)e<e.dd*Z=e/j7ej>e=e?e/sled+ej@dned,e?e/e.dZ.xKe.D]@ZAe-j1d-kreAe-_Be-jCd.e-jDd/ejEZFnejGeAZFeFjHZIe?e/ZJdgeJZKdgeJZLed0eMeFjNZOeOZPxueMeJD]gZQeFZRe/eQZSedd1dy eSjTZUWneVk rnXejWeUe-jXse-jYre-jXZZdZ[ej\eRd2e-jXd3e-n e]Z[e]ZZe^eSd4r eSj_r eZ r ed5ed6ed5ne^eSd7r`eSj`r`e[ r`ed5ed8ed5ne"eSe-eRd9d\ZPZOeeOeReSd:e-jaeKeQre-jse^e-d?re-jrgZndZeeOeReSeecd9e-jvd=esd@edAedBed:e-ja\ZwZxZyZzZ{Z|Z}eqdkr3ejde}jejeZqne-jr[ xeD]\ZZZZeOeZe^e-dCre-jldkree-jlkreojeddeoer e-jr xeD]t\ZZZZeOeZe-jldkr ee-jlkr enjeddener2 e-jr2 dEGHdFGHxeD]~\ZZZZeReOeZeSj~s edeks tdGe9ede9efdHe9edeefGHq WxeD]~\ZZZZeReOeZeSj~s edeks tdGe9ede9efdIe9edeefGHq WxeD]~\ZZZZeReOeZeSj~s edeks tdGe9ede9efdJe9edeefGHq WdKGHnewefeQre-jrenjZe?eredjx=eD]2ZeFeZedie9edenefqWqqqWqWndS(ks3 command line utility for screening composite models **Usage** _ScreenComposite [optional args] modelfile(s) datafile_ Unless indicated otherwise (via command line arguments), _modelfile_ is a file containing a pickled composite model and _filename_ is a QDAT file. **Command Line Arguments** - -t *threshold value(s)*: use high-confidence predictions for the final analysis of the hold-out data. The threshold value can be either a single float or a list/tuple of floats. All thresholds should be between 0.0 and 1.0 - -D: do a detailed screen. - -d *database name*: instead of reading the data from a QDAT file, pull it from a database. In this case, the _datafile_ argument provides the name of the database table containing the data set. - -N *note*: use all models from the database which have this note. The modelfile argument should contain the name of the table with the models. - -H: screen only the hold out set (works only if a version of BuildComposite more recent than 1.2.2 was used). - -T: screen only the training set (works only if a version of BuildComposite more recent than 1.2.2 was used). - -E: do a detailed Error analysis. This shows each misclassified point and the number of times it was missed across all screened composites. If the --enrich argument is also provided, only compounds that have true activity value equal to the enrichment value will be used. - --enrich *enrichVal*: target "active" value to be used in calculating enrichments. - -A: show All predictions. - -S: shuffle activity values before screening - -R: randomize activity values before screening - -F *filter frac*: filters the data before training to change the distribution of activity values in the training set. *filter frac* is the fraction of the training set that should have the target value. **See note in BuildComposite help about data filtering** - -v *filter value*: filters the data before training to change the distribution of activity values in the training set. *filter value* is the target value to use in filtering. **See note in BuildComposite help about data filtering** - -V: be verbose when screening multiple models - -h: show this message and exit - --OOB: Do out an "out-of-bag" generalization error estimate. This only makes sense when applied to the original data set. - --pickleCol *colId*: index of the column containing a pickled value (used primarily for cases where fingerprints are used as descriptors) *** Options for making Prediction (Hanneke) Plots *** - --predPlot=: triggers the generation of a Hanneke plot and sets the name of the .txt file which will hold the output data. A Gnuplot control file, .gnu, will also be generated. - --predActTable= (optional): name of the database table containing activity values. If this is not provided, activities will be read from the same table containing the screening data - --predActCol= (optional): name of the activity column. If not provided, the name of the last column in the activity table will be used. - --predLogScale (optional): If provided, the x axis of the prediction plot (the activity axis) will be plotted using a log scale - --predShow: launch a gnuplot instance and display the prediction plot (the plot will still be written to disk). *** The following options are likely obsolete *** - -P: read pickled data. The datafile argument should contain a pickled data set. *relevant only to qdat files* - -q: data are not quantized (the composite should take care of quantization itself if it requires quantized data). *relevant only to qdat files* i(tRDConfig(t DataStructsN(tImaget ImageDrawii(t DataUtilst SplitData(t CompositeRun(t DbConnect(tDbModules3.3.0cCs5|rtjjd|ntjjd|dS(s emits messages to _sys.stdout_ override this in modules which import this one to redirect output **Arguments** - msg: the string to be displayed s%s s%s N(tsyststdouttwrite(tmsgtnoRet((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytmessages cCstjjd|dS(s emits messages to _sys.stderr_ override this in modules which import this one to redirect output **Arguments** - msg: the string to be displayed s ERROR: %s N(R tstderrR (R ((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pyterrors cCs|dks||jdkr#dSttt|}tt|dd|f}|r|||f|}tt||ddf}||}ndS||S(Nig(tshapetfloattsum(tmatttgttnPtstnTgtPredt pctCorrecttnTgtRealt pctOverall((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytCalcEnrichments"" cCsxtt|D]~}|j|}t|drt|jtjkri}t|drx|jD]} d|| R<R=((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytDetailedScreens".       c &Cst|} | dkr!g} n| dkr6g} n| dkrKg} nt||||d|d| d| d| d|d|d| t| }t| }||}|rdGHd || d t|| ||d t||fGHnt| }|d kro|r0d || d t|| fGHntjg| D]}|d ^q=}t|t|}nd}|d krtjg| D]}|d ^q}t|t|}nd}|d kr.g| D]}|d^q}tjg| D]}|d ^q}t|t|}ng}g}d}|r\Hd|GHd|GHntj||ftj}x$|D]}|||fcd7sR1iRNR/R0RCRDREt errorAnalysist enrichTgtRiN()R&R"t GetDataSetRRtExplicitBitVectRAR#R$tListTypet TupleTypeR R}RHRJRRt _randomSeedR{RtInitRandomNumbersRRtTruetFalsetRandomizeActivitiesRRRRtGetQuantBoundsR1RcRRRR'RRtsqrttargsort(8tmodelsRR/tsetupR0RCRDRER@t enrichmentsRdtnModelsRQRPRStconfGoodtconfBadtconfSkipR\t badVoteDictt noVoteDictR7RtseedRt randomizeRRt nPossibletcbR1tgtbRttaGtaBtaStvTtatptcR8tlabeltavgNBadtdevNBadtbestIdxtavgNGoodtdevNGoodtavgNSkiptdevNSkipt avgConfBadt devConfBadt avgConfGoodt devConfGoodt avgConfSkipt devConfSkip((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytScreenFromDetails{s9                                    !          %%%%%%cCsts dSy$t|t|t|}WnOtk r|d}|d}|d}t|t|t|}nX|sdSd }d }d }t||}t||} t||} |dkrd }ntjd|d } tj| } dd|dd|ddf} d}t||d}| j | ||d||}t|| d}| j | ||d||}t|| d}| j | ||d|| S(NiiditRGBiiihtfill(ididi(iidid(iiid(idid(iii( thasPilR&Rt TypeErrorRtnewRtDrawRKtpieslice(RQRPtnRejtsizetnTott goodColortbadColortrejColortpctGoodtpctBadtpctRejtimgtdrawtboxtstartPtendP((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytGetScreenImageLs@$    $  "t.c Cst|tjkrd} nd} |rCdg} | jdng} | jd| st|||} | r|rdj|df}nd}| j|| jd|qnt|}tj |d}tj |d}tj |tj }tj |tj }xrt |D]d}||rXt |||f||||s&

VOTE DETAILS

st/s votes.pngs
s
s s ssssss s ssss)s
%d% Accurate
%d%d%.2f%d%.2f%4.2fgY@s"Predicted
% Pure%4.2f
Original
gs1

%d of %d examples were misclassified (%%%4.2f)s.

%d of %d overall: (%%%4.2f)s,

%d of %d examples were rejected (%%%4.2f)sG

The correctly classified examples had an average confidence of %6.4fsI

The incorrectly classified examples had an average confidence of %6.4fs;

The rejected examples had an average confidence of %6.4fsH

%.2f(%.2f) of %.2f(%.2f) examples were misclassified (%%%4.2f(%4.2f))s=

%.2f(%.2f) of %d overall: (%%%4.2f(%4.2f))s;

%.2f(%.2f) of %d examples were rejected (%%%4.2f(%4.2f))sM

The correctly classified examples had an average confidence of %6.4f(%.4f)sO

The incorrectly classified examples had an average confidence of %6.4f(%.4f)sA

The rejected examples had an average confidence of %6.4f(%.4f)ss 

Definitions:

  • % Pure: The percentage of, for example, known positives predicted to be positive.
  • % Accurate: The percentage of, for example, predicted positives that actually are positive.
ss (R#R$RR(RtjointsaveR RHRRJRRtxrange(RQRPRR[RXRVt voteTabletimgDirtfullPagetskipImgt includeDefst multModelstoutTxtRt imgFileNametnPosst pureCountst accCountstpureVecttaccVectR7R2tnTotaltnClasstpctErrRtdevClasst devPctErrt devPctRejttxt((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pyt ScreenToHtmlts      % )               "   "    c&Cst|d s|j rdS|r1tdnt|jd}td|jd} g|D]} || |^q`} t|j|jd|jd|j} | j } | |}t|d s|j s|j |jkr| }n't|j|j d|jd|j}|r&td ng}t | d t d t d gkrvg| D]} t | ^q[} nt jgt| }t|d r|jr|j}n|j d}d|dj|f}|jdd||fd|d| }|rtdndgt| }x3|D]+}|\}}| j|}||| Constructing Prediction (Hanneke) Plotsw+s%s.gnutusertpasswordt predActTables -> Pulling Activity DataiRGut predActColis %s in (%s)t,tfieldss%s,%stwheretextrass -> Creating Plots#ID Pred Conf %s R&s%s %d %.4f %f t predLogScaleslog(%s)t_t s# Generated by ScreenComposite.py version: %s set size square 0.7 set yrange [:1] set data styl points set ylab 'confidence' set xlab '%s' set grid set nokey set term postscript enh color solid "Helvetica" 16 set term X s'%s' us 4:($2==%d?$3:0/0)s#'%s' us (log10($4)):($2==%d?$3:0/0)splot %s s # EOF tpredShow(tGnuplotscd "%s"s load "%s.gnu"spress return to continue... ($R"RRtopenRtdbNamet tableNametdbUsert dbPasswordtGetColumnNamesRR#tstrRt placeHolderR RRtGetDataR&tindexR RtcloseRtreplacet__VERSION_STRINGRR(RtosRtgetcwdt raw_inputt tracebackt print_exc(&RR,R@RCRDtnRestidColRNtoutFtgnuFRTtptIdstorigConntcolNamestidNametactConntptstwhereLt actColNametwhereTxttrawDtactsRbtidtactR8R]R<R=tactLabeltgnuHdrtplotsR7tgnuTailRRRR((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pyt MakePredPlots !     ("      %  %          cCsdS(N((R((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytGoscCsh|dkrt}ntj|dg|_d|_d|_d|_d|_d|_ d|_ |S(Ngi( R&t_detailsRt SetDefaultst screenVoteToltdetailedScreenRrRsRRNRd(R((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pyR/s          cCstGHtjddS(s\ prints a list of arguments for when this is used from the command line and then exits iN(t__doc__R texit(((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytUsagescCs8dtGH|r4ddl}dGHdj|jGHndS(s- prints the version number of the program s%This is ScreenComposite.py version %siNscommand line was:R(RR Rtargv(t includeArgsR ((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pyt ShowVersions   c Csddl}y>|jtjdddddddd d d g\}}Wn$ddl}|jtnXd }d |_d |_d |_d |_ d |_ d |_ d|_ d|_ x|D]\}}|dkr||_q|dkrd|_q|dkrd|_t|}t|tgtd&gkrS|g}nx@|D]8} | dksx| d krZtdtjdqZqZW||_q|dkr||_q|dkrd |_d|_q|dkrd |_d|_q|dkrd|_d|_q|dkr>d|_d|_q|dkrVd|_q|dkrnd|_q|dkrtq|dkrt||_q|dkrt||_q|dkrd} q|dkrd|_||_q|dkr||_q|d kr&||_q|d!kr>d|_ q|d"krVd|_ q|d"krnd|_ q|d#krd|_ q|d$krt |d|_ q|d%krt ||_ qtqWt!|dkrtn|S('NiisEDd:t:VN:HThSRF:v:AXs predPlot=s predActCol=s predActTable=RRtOOBs pickleCol=senrich=RGis-ds-Ds-ts(Voting threshold must be between 0 and 1is-Ns-Hs-Ts-Es-As-Ss-Rs-hs-Fs-vs-Vs --predPlots --predActCols--predActTables--predLogScales --predShows--OOBs --pickleCols--enrich(ii("tgetoptR R5RRR4RRRRRR1RRRR1RdtevalR#RR3R0tnoteRsRrRtshowAllRRRRvRRKR ( RR9targsRRtfNametargtvalRettolRN((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pyt ParseArgss                    $                                               t__main__R6s"-> Retrieving models from databaseRRRswhere note='%s'sModel load faileds-> Loading modeltrbsNo composite models founds-> Working with %d models.RGRRs'-> Constructing and screening data setsR RRRs'*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*sD****** WARNING: Shuffled model being screened with unshuffled data.RsB****** WARNING: Random model being screened with non-random data.RNR1sC -----*****-----*****-----*****-----*****-----*****-----*****----- s Tolerance: %fRBR<RRDRERCRRs1-v-v-v-v-v-v-v- All Votes -v-v-v-v-v-v-v-s>id, prediction, confidence, flag(-1=skipped,0=wrong,1=correct)sbad point?: %s != %ss%s, %d, %.4f, 1s%s, %d, %.4f, 0s%s, %d, %.4f, -1s -^-^-^-^-^-^-^- -^-^-^-^-^-^-^-s0-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*s AVERAGES:is9Misclassifications: %%%5.2f(%%%5.2f) %4.1f(%4.1f) / %dids1 threshold: %%%5.2f(%%%5.2f) %4.1f(%4.1f) / %ds0Number Skipped: %%%4.2f(%%%4.2f) %4.2f(%4.2f)s Confidences:s Correct: %4.2f(%4.2f)s Incorrect: %4.2f(%4.2f)s Skipped: %4.2f(%4.2f)sResults Table:s s % 6.2fs | % 4.2fgY@s-------s& Enrichment of value %d: %.4f (%.4f)s0------------------------------------------------s Best Model: s&Misclassifications: %%%5.2f %d / %ds threshold: %%%5.2f %d / %dsNumber Skipped: %%%4.2f %ds Correct: %4.2fs Incorrect: %4.2fs Skipped: %4.2fs2 *-*-*-*-*-*-*-*- ERROR ANALYSIS -*-*-*-*-*-*-*-* s ---> Bad Vote Countss ---> Skipped Compound Countss%s,%ds ---> Good Vote Counts(R2trdkitRRR tcPickleR$tcopyRHtPILRRt ImportErrorRt rdkit.ML.DataRRtrdkit.MLRtrdkit.Dbase.DbConnectionRt rdkit.DbaseRR.RRRRR&R?RFRcRoRqRRRRR,R-R/R4R7RBt__name__RRRR;RttblNametconnR tblobstblobR(tloadsR RRRt modelFiletloadR R3R>RRRRR@t BuildDataSett GetVarNamest descNamesRRBtdataSetsRR}RRtmodelIdxttmpDRRRR{RRRRRRRR"RRR1R0RARJRRQRPRSRRRRRt goodVoteDictRRR\t SetInputOrdert screenResR1RdRNRRRtRRRRR*RRRDRER<RRCRRRRRR8RR'R]R<R=tpttAssertionErrorRRRRRRRRRRRRRRRRRRRORLRMR_R`R7RaR2RbtmeantdevtbestBadtbestGoodtbestSkiptkeystkstk(((sZ/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/python2.7/site-packages/rdkit/ML/ScreenComposite.pytns0      I =   p  0   (w    Z                                          *    !      !  ! !"# # #       %%%%%%               (  .  !#                  (  .           ,