xTc@ sddlmZddlmZddlmZddlmZddlm Z ddlm Z ddl m Z ddl m Z dd lmZdd lZdd lmZdd lmZdd lmZdd lZdd lZdefdYZdefdYZdefdYZdefdYZeddZ dej!fdYZ"dej!fdYZ#defdYZ$d S(i(tdivision(tscaling(txray_scattering(tflex(tadptbx(tuctbx(tchebyshev_polynome(tchebyshev_lsq_fit(t eigensystemN(tSorry(t format_value(t table_utilst gamma_proteincB seZddgZdZRS(tgammat sigma_gammac -C sddddddddd d d d d ddddddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-g-}tj|}d.d/d0d1d2d3d4d5d6d7d8d9d:d;d<d=d>d?d@dAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZg-}tj|}d[}d\}td]|||}td]|||}|j}||k}|j||}||k} |j| |}tj||ksttj||kst|j||_ |j||_ dS(^NgOϿg@b?gdݯ@o?gGi62?gjRaCgMv?g0Rx9g-ڰ84?g|:l?g-jgm/ĝ?g߶zzg#-0?gmJ{glwg`ۋ?gi|Fjg o'?gUIQg 4=/?g%ɂgI* {?gvxgC"7s::?g{9Eg܌zpgw?gb:I~g_ԷXgt.mgj84Uzgh6|3x?g5xfg8gm?gʡ1gsgzLMgw#O?g=/ttgb`/s?gr2lig/o?gݨ@OY?ghR?gYV^Y?g!]pglfҤ?g-\?gjÊ?gA@;-g!g;!1og T{[g0*lg)[=Ogdɼvng@|4?g Urg{gYLgM +ggI?g5ֿ^D?g}Mwn^?gTåTT?gP؃%-?gzG1gq٪YRgØ;KKgVRgI<zKg1X U)gvo%29g0!#d-g/cYNg5DgV&"5g@Zr:?gjNsԧW?gjwL1V?g7k[?gji4?W?gJ@U?g7L?gVvd?d@dAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZg-}tj|}d[}d\}td]|||}td]|||}|j}||k}|j||}||k} |j| |}tj||ksttj||kst|j||_ |j||_ dS(^Ngv/[ӿgqL??gRve?gB?gkrug"Zý?g9gN1?g HL?g gc^+O?gu=樂gC ?g^C?gbgG?gHWRg"Y?֐gz{?g !l gV?gx?g3'sg3 W;?g g?g7%or?gۢ|I9?g6N g³k?g75_?g3"0Jgϰs?g(sYn?gmXng7|?go8 gw@?gZJ]|gsagdcp?gƘO@qg`=GL]?gVa ?gȼQs?gZ?pk?g_ƹ?gM?gqc.?g޲?g9=v?g/i?g#0{geCH?g$Ԣq?gv&T{gyq?grg rdag[ģ^?gp'`@ug-8gӅ^gkwgQ"rag-ngL!%jg "#\?g~QcgA+"q?ggCR3l?g‹i?ga0n?gG N=Wgjgug0pgi zwgBtg7Lugb Qg^/vQGgS"oQP?g R!I?g0[6?ggka?g>L4ggXg<4?gtTWFgH>gMb?gGz?i-( RRRRRRRRRR R( RRRRRRRRRR((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR ps~     (R!R"R (((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR$ostexpected_intensitycB s eZdZddddZRS(sx This class computes the expected intensity for a given d_star_sq_array given some basic info about ASU contents. gg@cC s|j||j|_|jd|j|_|j|_|jtj| |d|_|jtj| 9_|j|_|j|j|_|jtj| |d|_|jtj| 9_dS(Ng?g@( t scat_datat sigma_tot_sqtmean_intensityt gamma_totRtexptmathtgamma_tot_sigmatsigma_intensity(Rtscattering_infotd_star_sq_arraytp_scaletb_wilsontmagic_fudge_factor((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR s     (R!R"t__doc__R (((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR%stscattering_informationcB s,eZddddddZddZRS(cC s5d|_d|_d|_|dk rN|dks9t|dksNtn|dk sf|dk r{|dks{tn|dk r|dk st|dk stn|dkrA|dkrd}nddddt|}|dkrd}ndd d"d$d&t|}||}||}||}n|dk rY||_n d|_|dk rz||_n d|_|j|jdkst|dkr(d}|dkr idt|d6dt|d6dt|d6dt|d6}n|dkrtit|d d6t|d d6t|d d6t|d d6t|dd6}n|dkr(|dkr%it|d dt|d6t|d dt|d6t|d dt|d6t|d dt|d6t|dd6}q%q(n||_dS('s0 Returns scattering info for specified structuregg @g?g@g@g?g@g333333?g0@gffffff#@gffffff@g@g.@itHtCtNtOtPNg @g @g>@gf@g%@g`R@g333333#@g333333S@g0@g0@gM@g33333u@g:@gfffffFg@gG@gw@g.@g l@( tNoneR'R)R,Rtfloattfraction_proteintfraction_nucleict asu_contents(Rt n_residuestn_basesR>R<R=ttot((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR sf                         "cC s|dkr*d|_d|_d|_n|dk rtj|j|_i}x`|jjD]O\}}t j |j ||<||j |}|j|||7_qdW|jdkrt |}|j|j|_t|}|j|j|_|j|j|j|j|_|j|j|j|j|j|j|j|j|_tj|j|_qndS(Ni(R:R'R,R)RRtsizeR>titemsRtwk1995tfetcht at_d_star_sqR R R<t gamma_protR$R=t gamma_nucRtsqrt(RRt gaussianst chemical_typetn_atomstf0RGRH((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR&:s0          N(R!R"R:R R&(((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR4s  Qgc C sd}y%|jj|jj|k}Wn$tk rAntk rQnX|dkr||j|j|k}nt}|jr|j}t }n|j st |dk rt j |j |}||}ntj|j|j||j |}|jdk rVtj|j|j||j |} nd} |jd|d| j|}|rtj|j} | tjtjjkstdd| n|j}n|S(Ntdatatsigmass/Amplitudes will exceed floating point limit if s&converted to intensities (max F = %e).(R:tinputtselectRNtKeyboardInterruptt ExceptiontFalsetis_xray_intensity_arrayt f_sq_as_ftTrueRRtb_iso_as_u_start unit_cellRtml_normalise_anisotindicesROtcustomized_copytset_observation_typeRRR+RItsyst float_infot OverflowErrort f_as_f_sq( tcache_0R0tu_startb_addtmust_be_greater_thant work_arraytchange_back_to_intensityt u_star_addtcorrected_amplitudestcorrected_sigmastf_max((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pytanisotropic_correction]sP%                tml_iso_absolute_scalingcB sDeZdZdddddedZdZdZdZRS(s Maximum likelihood isotropic wilson scaling. :param miller_array: experimental data (will be converted to amplitudes if necessary :param n_residues: number of protein residues in ASU :param n_bases: number of nucleic acid bases in ASU :param asu_contents: a dictionary specifying scattering types and numbers ( i.e. {'Au':1, 'C':2.5, 'O':1', 'H':3 } ) :param prot_frac: fraction of scattering from protein :param nuc_frac: fraction of scattering from nucleic acids g?gc C sd\|_|_|dkr`|dkr`|dk s<tt|tiks]tq`n|dkr|dk s|dk stn||dkst|jr|j}n|jstd|_|r|j|_n|j ddt j t j ddt j t j}|jjdkr|j|jdk}|jj|_d|_|dkrtd|d|d|d||_ntd |d|d||_|jdkr|jj|j|j|_|jdk r)|j|_ntj|jjd |_tj|jdkrr|jd |_n|jjj|_tj|j j|_!|jj"|_#|jj$|_%tjd d |_&d |j&dgigI@itmax_iterationsg.Attarget_evaluatorttermination_params(NN(,R:R0R1RttypeRURVt is_real_arraytinfotresolution_filterR+RIRtget_d_star_sq_low_limittget_d_star_sq_high_limitRNRBRQRt scat_infoR4R&tf_obsROt sigma_f_obsRRRtepsilonst as_doubletepsilontboolt centric_flagstcentricR)RGR't sigma_prot_sqtxRtscitbxtlbfgsttermination_parameterstrun( Rt miller_arrayR?R@R>t prot_fractnuc_fractinclude_array_infoRftterm_parameterst minimizer((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR sx  $ !          cC stjd|jd|jd|jd|jd|jd|jd|jd|j d d |j d  }t j tj d|jd|jd|jd|jd|jd|jd|jd|j d d |j d  }||_ ||fS( NRRzR{R~tsigma_sqRGRR0it p_B_wilsoni(Rtwilson_total_nllRRzR{R~RRGRRRRtwilson_total_nll_gradientR(RRtg((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyt compute_functional_and_gradientss,                  cC sd}|jdk r+dt|j}n|jd|jd||jdtd|j|jd||jdtd|j|jd |jd kr|j d d d dddndS(Nts of %ss+Maximum likelihood isotropic Wilson scalings" ML estimate of overall B value%s:s %s A**(-2)s%5.2fs" Estimated -log of scale factor%s:s %ss The overall B value ("Wilson B-factor", derived from the Wilson plot) gives an isotropic approximation for the falloff of intensity as a function of resolution. Note that this approximation may be misleading for anisotropic data (where the crystal is poorly ordered along an axis). The Wilson B is strongly correlated with refined atomic B-factors but these may differ by a significant amount, especially if anisotropy is present.is6The Wilson B-factor is negative! This indicates that s9the data are either unusually pathological, or have been s>artifically manipulated (for instance by applying anisotropic s;scaling directly). Please inspect the raw data and/or use s@unmodified reflections for phasing and refinement, as any model s7generated from the current dataset will be unrealistic.( RuR:tstrtshow_sub_headertshowtshow_preformatted_textR R1R0twarn(Rtoutt label_suffix((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyt _show_impls    cC s0|jdkrdgS|jdkr,d gSgS( Niis/The Wilson plot isotropic B-factor is negative.s+Maximum likelihood isotropic Wilson scalingiis7The Wilson plot isotropic B-factor is greater than 200.(is/The Wilson plot isotropic B-factor is negative.s+Maximum likelihood isotropic Wilson scaling(is7The Wilson plot isotropic B-factor is greater than 200.s+Maximum likelihood isotropic Wilson scaling(R1(R((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pytsummarize_issues,sN( R!R"R3R:RWR RRR(((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyRms  I  tml_aniso_absolute_scalingcB s}eZdZddddddZdZdZdZddZd Z d d d d dZ dZ dZ RS(s Maximum likelihood anisotropic wilson scaling. :param miller_array: experimental data (will be converted to amplitudes if necessary :param n_residues: number of protein residues in ASU :param n_bases: number of nucleic acid bases in ASU :param asu_contents: a dictionary specifying scattering types and numbers ( i.e. {'Au':1, 'C':2.5, 'O':1', 'H':3 } ) :param prot_frac: fraction of scattering from protein :param nuc_frac: fraction of scattering from nucleic acids g?gc C s|dkrN|dkrN|dk s*tt|tiksKtqNn|dkr{|dk s{|dk s{tn||dkst|jst|j|_|jr|j}n|jddtj t j ddtj t j }|j |jdk}|jj|_d|_|dkrktd|d||_ntd|d|d ||_|jj|jd|_|jdkr|j|_|j|_tj|jj|_|jdk r|j|_ntj|jjd |_tj|jdkr`|jd |_n|j jj!|_"|jj#|_$|jj%|_&tj'|j(j|_)|j*|_+|j+j,|_,|j,j-|_.|j.d ksttj|j.d d |_/t0j1j2d t3dt3dt3}t0j1j4dd} t0j1j5d|d| d|} tj6|jj7d} |j/d|_8|j9|_:t;tj|j:| |_:t<j=t<j>|j|j:|_t<j?|j|j:|_@tAjB|j} | jC|_D| jE|_F||_G|jH|jDddkrtI|jDd|jDd|jDd|_Jn d|_J|`/|`|`|`"|`$|`&|`)|`|`|`,ndS(s. Maximum likelihood anisotropic wilson scalingg?RnRoiR?R@R>R<R=giit-ignore_line_search_failed_step_at_lower_boundt-ignore_line_search_failed_step_at_upper_boundt ignore_line_search_failed_maxfevRpi2RqRrtexception_handling_paramsg@g@iNgUUUUUU?(KR:RRsRtRuRURVRvR+RIRRwRxRQRNRRyR4R&tb_cartRBR[thklRzRRYt parametersROR{RRRR|R}R~R)RGR'RRRRt space_grouptsgtadp_constraintstn_independent_paramstdim_uRRRtexception_handling_parametersRTRRtpowtvolumeR0tunpackRctlistRtu_as_btu_star_as_u_carttu_star_as_u_ciftu_cifRtreal_symmetrictvaluest eigen_valuestvectorst eigen_vectorsRftanalyze_aniso_correctiontabstanirat( RRR?R@R>RRRfRRRtVrwgkteigen((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR Bs  $ !               cC s@|dg}|t|jjt|d}tj|S(Nii(RRtindependent_gradientsRR(RRt g_independent((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pytpacks   cC s#|jjt|jd}|S(Ni(Rt all_paramsRR(Rt u_star_full((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyRsc C s|j}tj|j|j|j|j|j|j|j |j d|j | }||_ t jtj|j|j|j|j|j|j|j |j d|j | }|j|}||fS(Ni(RRtwilson_total_nll_anisoRRzR{R~RRGRRRYRRRtwilson_total_nll_aniso_gradientR(RtuRt g_full_exactR((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyRs2        s%3.2fcC s'||}|dkr#d|}n|S(Nit ((RRtformattxx((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyt format_its   c C sdStjdddddddd d d d g }tjjd dd|}tjtdd}t|d}tjtj|j ||d}|d||}tjtj|j ||d|d}tj || S(NigHiiitreverse($R:tmin_dtmax_dtleveltz_leveltz_lowtz_hightz_tott mean_isigit mean_countt mean_isigi_low_correction_factortfrac_below_low_correctiont!mean_isigi_high_correction_factortfrac_below_high_correctionRfROR\RNRlRcRaRRRt d_spacingsR}RBtsort_permutationRQtmeanRRWRR+RIR(Rtn_checktp_checkRRtcorrection_factorstisigiRtd_sorttd_selectRRRtobserved_countRt isigi_ranktcorrection_ranktn_againtsel_hctsel_lcRRRRtmutvarRRR((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyRsz                /               c C sj|jdk st|jd|jd|jd|jd|jd|jd|jd|jd|jd f|jd |jd|jd|jd|jd|jd|jd|jd f|jd d d }d||jddd||jd||jd||jd fgd||jddd||jd||jd||jdfgd||jd dd||jd||jd||jdfgg}t j ddddgd|}|j ||jd|jd|j |jd|jddkrot ddd d!nd"}|jdk r|j|jkr|j|jkrd#}qd$}n|jdk rf|jd%|j|j|jd&|jf|jd'|jd(|j|j|jf|jd)|jd(|j|j|jf|jd*ndS(+Ns-Maximum likelihood anisotropic Wilson scalings$ML estimate of overall B_cart value:s/ %5.2f, %5.2f, %5.2f %12.2f, %5.2f %19.2f iiiiiis#Equivalent representation as U_cif:sEigen analyses of B-cart:s%3.2fcS s'||}|dkr#d|}n|S(NiR((RRR((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyRGs   t1s%5.3fs (%s, %s, %s)t2t3iiitcolumn_headerst EigenvectortValuetVectort table_rowss%ML estimate of -log of scale factor:s %5.2fsAnisotropy analysess6Fatal error: eigenvector 1 of the overall anisotropic sB-factor B_cart is zero. This s?may indicate severe problems with the input data, for instance s;if only a single plane through reciprocal space is present.sPindicates that there probably is no significant systematic noise amplification.sindicates that there probably is significant systematic noise amplification that could possibly lead to artefacts in the maps or difficulties in refinementsindicates that there probably is some systematic dependence between the anisotropy and not-so-well-defined intensities. Because the signal to noise for the most affected intensities is relatively good, the affect on maps or refinement behavior is most likely not very serious.sx For the resolution shell spanning between %4.2f - %4.2f Angstrom, the mean I/sigI is equal to %5.2f. %4.1f %% of these intensities have an I/sigI > 3. When sorting these intensities by their anisotropic correction factor and analysing the I/sigI behavior for this ordered list, we can gauge the presence of 'anisotropy induced noise amplification' in reciprocal space. gY@sO The quarter of Intensities *least* affected by the anisotropy correction show s_ : %5.2e Fraction of I/sigI > 3 : %5.2e ( Z = %8.2f )sO The quarter of Intensities *most* affected by the anisotropy correction show s Z-scores are computed on the basis of a Bernoulli model assuming independence of weak reflections with respect to anisotropy.(RR:RRRRRRRR t simple_tablet show_tableR0R RRRRRRRRRRRRR(RRRtrowsttabletmessage((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR3sz  ! !         !  'cC st|jdd!d}t|jdd!t|jdd!}||}|dkrm|dkrmd gS|dkr|dkrd gSd gSdS(Niiiiis The data show severe anisotropy.s-Maximum likelihood anisotropic Wilson scalingg?s$The data are moderately anisotropic.s+The data are not significantly anisotropic.(is The data show severe anisotropy.s-Maximum likelihood anisotropic Wilson scaling(is$The data are moderately anisotropic.s-Maximum likelihood anisotropic Wilson scaling(is+The data are not significantly anisotropic.s-Maximum likelihood anisotropic Wilson scaling(RRRR(Rt b_cart_meant b_cart_ranget aniso_ratio((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyRs* N( R!R"R3R:R RRRRRRRR(((sL/public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/mmtbx/scaling/absolute_scaling.pyR5s  j      A mtkernel_normalisationcB s&eZdddddeddZRS(ii i2c  C s |dkr!|tk s!tn|tk rB|dksBtn|jdksZtd} |js{tdn|jr|j} n|jr|j } | j |} n|js|jstdqn| j } | j j j } | jj } d| | } |dkrLtj| }n|dkrjtj| }n||_|tk rtj| } |tkr|}n t|}|| jkr| jd}n| | |||_|jdkstn|dksttjd|d|d|d t|_tjd | d | d | d |jd|j|_tjd | d | | d | | d |jd|j|_|j|j|j|_tjd | d | ddd | ddd |jd|j|_|_d}|j|k|_|jj}|jdksKt|j|jjdkrt ddddn|jj!||_|jj!||_tj"|jj!||_|jj!||_tj"|j|_tj||j|j}t#||||j$|_%tj||j|j}t#||||j$|_&tj||j|j}t#||||j$|_tj'|j|_tj'|j%j(| |_)tj'|j|_tj'|j&j(| |_&tj'|jj(| |_d|_*d|_+| j,dk r| j-d| j |j)d| j,|j)j | |_*|j*j-d|j*j | d|j*j,| j | |_+nY| j-d| j |j)j | |_*|j*j-d|j*j | j | |_+dS(NisPlease provide real arrays onlysObservation type unknowng?itnRRtinclude_limitst d_star_sq_hkltI_hklR~R/t kernel_widthggؗҜs0   [V} ;{