ó ¦xxTc@€sÒdZddlmZdZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z mZmZddlmZdd lmZmZdd lmZdd lmZed „Zd efd„ƒYZd„Zd„Zd„Zd„Z ej!ddfkr[ddl"Z"e"j#Z$d„Z%d„Z&ne#Z$e Z%e Z&d„Z'defd„ƒYZ(de)fd„ƒYZ*defd„ƒYZ+de*fd„ƒYZ,dS( sÛRefactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. iÿÿÿÿ(twith_statements#Guido van Rossum N(tchaini(tdriverttokenizettoken(t find_root(tpytreetpygram(t btm_utils(t btm_matchercC€sœt|ggdgƒ}tjj|jƒ}g}xbttj|ƒƒD]K}|jdƒrI|jdƒrI|r€|d}n|j |d ƒqIqIW|S(sEReturn a sorted list of all available fix names in the given package.t*tfix_s.pyiiýÿÿÿ( t __import__tostpathtdirnamet__file__tsortedtlistdirt startswithtendswithtappend(t fixer_pkgt remove_prefixtpkgt fixer_dirt fix_namestname((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pytget_all_fix_names!s t _EveryNodecB€seZRS((t__name__t __module__(((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR.scC€sßt|tjtjfƒrC|jdkr3t‚nt|jgƒSt|tjƒrt|j rkt |j ƒSt‚nt|tj ƒrËtƒ}x5|j D]*}x!|D]}|j t |ƒƒq¦Wq™W|St d|ƒ‚dS(sf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. s$Oh no! I don't understand pattern %sN(t isinstanceRt NodePatternt LeafPatternttypetNoneRtsettNegatedPatterntcontentt_get_head_typestWildcardPatterntupdatet Exception(tpattrtptx((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR(2s      cC€stjtƒ}g}x£|D]›}|jr„yt|jƒ}Wntk r^|j|ƒq·XxU|D]}||j|ƒqfWq|jdk rª||jj|ƒq|j|ƒqWx:t t j j j ƒt j jƒD]}||j|ƒqÝWt|ƒS(s^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N(t collectionst defaultdicttlisttpatternR(RRt _accept_typeR$RRtpython_grammart symbol2numbert itervaluesttokenstextendtdict(t fixer_listt head_nodesteverytfixertheadst node_type((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt_get_headnode_dictNs"    cC€s(gt|tƒD]}|d|^qS(sN Return the fully qualified names for fixers in the package pkg_name. t.(RtFalse(tpkg_nametfix_name((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pytget_fixers_from_packagegscC€s|S(N((tobj((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt _identitynsiicC€s|jddƒS(Nu u (treplace(tinput((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt_from_system_newlinesuscC€s*tjdkr"|jdtjƒS|SdS(Ns u (R tlinesepRI(RJ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt_to_system_newlineswsc€sàt}tjtj|ƒjƒ‰‡fd†}ttjtjtj fƒ}t ƒ}ykxdt rÀ|ƒ\}}||kr„q]q]|tj kr¦|rPnt }q]|tj kr¼|dkr¼|ƒ\}}|tj ksë|dkrïPn|ƒ\}}|tj ks|dkrPn|ƒ\}}|tjkrY|dkrY|ƒ\}}nxa|tj kr¸|j|ƒ|ƒ\}}|tjks¢|dkr¦Pn|ƒ\}}q\Wq]Pq]WWntk rÕnXt|ƒS(Nc€sˆjƒ}|d|dfS(Nii(tnext(ttok(tgen(s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pytadvance…s ufromu __future__uimportu(u,(RCRtgenerate_tokenstStringIOtreadlinet frozensetRtNEWLINEtNLtCOMMENTR%tTruetSTRINGtNAMEtOPtaddt StopIteration(tsourcethave_docstringRQtignoretfeaturesttptvalue((RPs</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt_detect_future_features‚sD       t FixerErrorcB€seZdZRS(sA fixer could not be loaded.(RRt__doc__(((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRfªstRefactoringToolcB€s!eZied6ed6ZdZdZddd„Zd„Zd„Z d„Z d„Z d „Z eed „Z eed „Zd „Zeed „Zd„Zed„Zd„Zd„Zdedd„Zdd„ZdZdZd„Zd„Zd„Zd„Zd„Zd„ZRS(tprint_functiontwrite_unchanged_filestFixR cC€sÄ||_|pg|_|jjƒ|_|dk rI|jj|ƒn|jdretj|_ n tj |_ |jj dƒ|_ g|_ tjdƒ|_g|_t|_tj|j dtjd|jƒ|_|jƒ\|_|_g|_tjƒ|_g|_g|_ x}t!|j|jƒD]f}|j"rT|jj#|ƒq2||jkrv|jj$|ƒq2||jkr2|j j$|ƒq2q2Wt%|jƒ|_&t%|j ƒ|_'dS(sÒInitializer. Args: fixer_names: a list of fixers to import options: an dict with configuration. explicit: a list of fixers to run even if they are explicit. RiRjRhtconverttloggerN((tfixerstexplicitt_default_optionstcopytoptionsR$R*Rt!python_grammar_no_print_statementtgrammarR5tgetRjterrorstloggingt getLoggerRmt fixer_logRCtwroteRtDriverRRlt get_fixerst pre_ordert post_ordertfilestbmt BottomMatchertBMt bmi_pre_ordertbmi_post_orderRt BM_compatiblet add_fixerRRAtbmi_pre_order_headstbmi_post_order_heads(tselft fixer_namesRrRoR>((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt__init__¶s<            c C€sÝg}g}x•|jD]Š}t|iidgƒ}|jddƒd}|j|jƒrr|t|jƒ}n|jdƒ}|jdjg|D]}|j ƒ^q”ƒ}yt ||ƒ} Wn't k rìt d||fƒ‚nX| |j |jƒ} | jr?|jtk r?||jkr?|jd|ƒqn|jd |ƒ| jd krn|j| ƒq| jd kr|j| ƒqt d | jƒ‚qWtjd ƒ} |jd| ƒ|jd| ƒ||fS(sInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. R RBiiÿÿÿÿt_tsCan't find %s.%ssSkipping implicit fixer: %ssAdding transformation: %stpretpostsIllegal fixer order: %rt run_ordertkey(RnR trsplitRt FILE_PREFIXtlentsplitt CLASS_PREFIXtjointtitletgetattrtAttributeErrorRfRrRyRoRYt log_messaget log_debugtorderRtoperatort attrgettertsort( R‰tpre_order_fixerstpost_order_fixerst fix_mod_pathtmodREtpartsR.t class_namet fix_classR>tkey_func((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR|ês8/ cO€s‚dS(sCalled when an error occurs.N((R‰tmsgtargstkwds((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt log_errorscG€s'|r||}n|jj|ƒdS(sHook to log a message.N(Rmtinfo(R‰R©Rª((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR›s cG€s'|r||}n|jj|ƒdS(N(Rmtdebug(R‰R©Rª((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRœs cC€sdS(sTCalled with the old version, new version, and filename of a refactored file.N((R‰told_texttnew_texttfilenametequal((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt print_output!scC€sPxI|D]A}tjj|ƒr5|j|||ƒq|j|||ƒqWdS(s)Refactor a list of files and directories.N(R Rtisdirt refactor_dirt refactor_file(R‰titemstwritet doctests_onlyt dir_or_file((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pytrefactor&s c C€sétjd}xÕtj|ƒD]Ä\}}}|jd|ƒ|jƒ|jƒxe|D]]}|jdƒ rWtjj|ƒd|krWtjj||ƒ} |j | ||ƒqWqWWg|D]} | jdƒs¿| ^q¿|(qWdS(sÄDescends down a directory and refactor every Python file found. Python files are assumed to have a .py extension. Files and subdirectories starting with '.' are skipped. tpysDescending into %sRBiN( R textseptwalkRœR RRtsplitextR—R¶( R‰tdir_nameR¸R¹tpy_exttdirpathtdirnamest filenamesRtfullnametdn((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRµ/s    cC€syt|dƒ}Wn'tk r<}|jd||ƒdSXztj|jƒd}Wd|jƒXt|dd|ƒ}t |j ƒƒ|fSWdQXdS(sG Do our best to decode a Python source file correctly. trbsCan't open %s: %siNR-tencoding(NN( topentIOErrorR¬R$Rtdetect_encodingRTtcloset_open_with_encodingRKtread(R‰R±tfterrRÈ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt_read_python_sourceCs cC€s|j|ƒ\}}|dkr%dS|d7}|r›|jd|ƒ|j||ƒ}|jsl||krˆ|j|||||ƒqþ|jd|ƒnc|j||ƒ}|jsÅ|rî|jrî|jt|ƒd |d|d|ƒn|jd|ƒdS( sRefactors a file.Nu sRefactoring doctests in %ssNo doctest changes in %siÿÿÿÿR¸RÈsNo changes in %s( RÑR$Rœtrefactor_docstringRjtprocessed_filetrefactor_stringt was_changedtunicode(R‰R±R¸R¹RJRÈtoutputttree((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR¶Ss  cC€s·t|ƒ}d|kr*tj|j_nzMy|jj|ƒ}Wn0tk ru}|jd||jj |ƒdSXWd|j|j_X||_ |j d|ƒ|j ||ƒ|S(sFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. RisCan't parse %s: %s: %sNsRefactoring %s( ReRRsRRtt parse_stringR+R¬t __class__Rtfuture_featuresRœt refactor_tree(R‰tdataRRbRØRÐ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRÔjs     cC€sÆtjjƒ}|ro|jdƒ|j|dƒ}|jsI||kr_|j|d|ƒqÂ|jdƒnS|j|dƒ}|js™|rµ|jrµ|jt |ƒd|ƒn |jdƒdS(NsRefactoring doctests in stdinssNo doctest changes in stdinsNo changes in stdin( tsyststdinRÎRœRÒRjRÓRÔRÕRÖ(R‰R¹RJR×RØ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pytrefactor_stdin…s c C€s›x-t|j|jƒD]}|j||ƒqW|j|j|jƒƒ|j|j|jƒƒ|jj|j ƒƒ}xçt |j ƒƒrcxÎ|jj D]À}||krœ||rœ||j dtjjdtƒ|jrþ||j dtjjƒnx[t||ƒD]F}|||kr9||j|ƒnyt|ƒWntk r]qnX|jr|||jkr|qn|j|ƒ}|r|j||ƒ}|dk rU|j|ƒx9|jƒD]+}|jsäg|_n|jj|ƒqÉW|jj|j ƒƒ}x?|D]4} | |kr6g|| t match_settnodetresultstnewt new_matchestfxr((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRÜ•sJ          .cC€sƒ|s dSxr|D]j}xa||jD]R}|j|ƒ}|r%|j||ƒ}|dk rw|j|ƒ|}qwq%q%WqWdS(sTraverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N(R#RïRðR$RI(R‰Rnt traversalRóR>RôRõ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRãäs    cC€s½|jj|ƒ|dkrB|j|ƒd}|dkrBdSn||k}|j||||ƒ|rŠ|jd|ƒ|jsŠdSn|r©|j||||ƒn|jd|ƒdS(sR Called when a file has been refactored and there may be changes. iNsNo changes to %ssNot writing changes to %s(RRR$RÑR³RœRjt write_file(R‰R°R±R¯R¸RÈR²((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRÓûs    cC€s¸yt|dd|ƒ}Wn*tjk rE}|jd||ƒdSXzGy|jt|ƒƒWn)tjk r‹}|jd||ƒnXWd|jƒX|jd|ƒt|_ dS(sÑWrites a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. twRÈsCan't create %s: %sNsCan't write %s: %ssWrote changes to %s( RÍR terrorR¬R¸RMRÌRœRYRz(R‰R°R±R¯RÈRÏRÐ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRùs s>>> s... c C€s‡g}d}d}d}d}x+|jtƒD]}|d7}|jƒj|jƒr²|dk r„|j|j||||ƒƒn|}|g}|j|jƒ} || }q.|dk r|j||j ƒsñ|||j j ƒdkr|j |ƒq.|dk r/|j|j||||ƒƒnd}d}|j |ƒq.W|dk rz|j|j||||ƒƒndj |ƒS(sËRefactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) iiu uN( R$t splitlinesRYtlstripRtPS1R9trefactor_doctesttfindtPS2trstripRR—( R‰RJR±tresulttblockt block_linenotindenttlinenotlineti((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRÒ(s:        c C€ssy|j|||ƒ}Wnutk r}|jjtjƒrmx*|D]}|jd|jdƒƒqGWn|jd|||j j |ƒ|SX|j ||ƒrot |ƒj tƒ}||d ||d} }| dg|dksøt| ƒ‚|djdƒs|dcd7>>" (possibly indented), while the remaining lines start with "..." (identically indented). s Source: %su s+Can't parse docstring in %s line %s: %s: %siiÿÿÿÿi(t parse_blockR+Rmt isEnabledForRwtDEBUGRœRR¬RÚRRÜRÖRüRYtAssertionErrorRRþtpopR( R‰RRRR±RØRÐRRõtclipped((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRÿSs&   #.cC€s|jrd}nd}|js4|jd|ƒn1|jd|ƒx|jD]}|j|ƒqNW|jrŸ|jdƒx!|jD]}|j|ƒq…Wn|jrt|jƒdkrÍ|jdƒn|jdt|jƒƒx0|jD]"\}}}|j|||ŽqðWndS( Ntweres need to besNo files %s modified.sFiles that %s modified:s$Warnings/messages while refactoring:isThere was 1 error:sThere were %d errors:(RzRR›RyRvR”(R‰RtfiletmessageR©RªR«((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt summarizeps$      cC€s1|jj|j|||ƒƒ}tƒ|_|S(s³Parses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. (Rt parse_tokenst wrap_toksRURÛ(R‰RRRRØ((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR ‡s! c c€sŠtj|j||ƒjƒ}xe|D]]\}}\}}\} } } ||d7}| |d7} ||||f| | f| fVq%WdS(s;Wraps a tokenize stream to systematically modify start/end.iN(RRRt gen_linesRN( R‰RRRR8R#Rdtline0tcol0tline1tcol1t line_text((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR‘s (cc€s¢||j}||j}|}xi|D]a}|j|ƒrN|t|ƒVn4||jƒdkrldVntd||fƒ‚|}q'WxtrdVqWdS(s–Generates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. u sline=%r, prefix=%rRN(RþRRR”RR RY(R‰RRtprefix1tprefix2tprefixR((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRŸs     N(RRRCRpR–R“R$R‹R|R¬R›RœR³R»RµRÑR¶RÔRàRÜRãRÓRùRþRRÒRÿRR RR(((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyRh®s:  4 (         O    +   tMultiprocessingUnsupportedcB€seZRS((RR(((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR³stMultiprocessRefactoringToolcB€s5eZd„Zeedd„Zd„Zd„ZRS(cO€s/tt|ƒj||Žd|_d|_dS(N(tsuperR R‹R$tqueuet output_lock(R‰Rªtkwargs((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR‹¹s ic C€sf|dkr(tt|ƒj|||ƒSyddl}Wntk rQt‚nX|jdk rptdƒ‚n|j ƒ|_|j ƒ|_ gt |ƒD]}|j d|jƒ^q›}z;x|D]}|jƒqÆWtt|ƒj|||ƒWd|jjƒx$t |ƒD]}|jjdƒqWx'|D]}|jƒr5|jƒq5q5Wd|_XdS(Niiÿÿÿÿs already doing multiple processesttarget(R!R R»tmultiprocessingt ImportErrorRR"R$t RuntimeErrort JoinableQueuetLockR#txrangetProcesst_childtstartR—tputtis_alive( R‰R·R¸R¹t num_processesR&R t processesR.((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR»¾s2    +     cC€so|jjƒ}xY|dk rj|\}}ztt|ƒj||ŽWd|jjƒX|jjƒ}qWdS(N(R"RuR$R!R R¶t task_done(R‰ttaskRªR$((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR-Ûs cO€sE|jdk r(|jj||fƒntt|ƒj||ŽSdS(N(R"R$R/R!R R¶(R‰RªR$((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR¶æs(RRR‹RCR»R-R¶(((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyR ·s    (-Rgt __future__Rt __author__R RÞRwRžR0RSt itertoolsRtpgen2RRRt fixer_utilRRRRRtbuR R€RYRR+RR(RARFRHt version_infotcodecsRÉRÍRKRMReRftobjectRhRR (((s</opt/xtal/ccp4-6.5-linux32/lib/python2.7/lib2to3/refactor.pyt sF                (ÿÿ