ó ä™xTc@ sìdZddlmZddlmZdefd„ƒYZd„Zd„Zd„Z d d „Z e dd „Z d „Zd „Zd„Zed„Zddd„ƒYZddd„ƒYZdefd„ƒYZddd„ƒYZdS(s- http://pypi.python.org/pypi/cluster/1.1.1b3 iÿÿÿÿ(tdivision(t TupleTypetClusteringErrorcB seZRS((t__name__t __module__(((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRscC sKt|ƒtgƒkr|gS|gkr/|St|dƒt|dƒS(sR Flattens a list. Example: flatten([a,b,[c,d,[e,f]]]) = [a,b,c,d,e,f] ii(ttypetflatten(tL((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRs  cC sWt|ƒ}|}|jƒ|d@r3||dS||dd||ddSdS(s~Return the median of the list of numbers. found at: http://mail.python.org/pipermail/python-list/2004-December/253517.htmliig@N(tlentsort(tnumberstntcopy((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pytmedian&s     cC s tt|ƒƒtt|ƒƒS(sƒReturns the arithmetic mean of a numeric list. found at: http://mail.python.org/pipermail/python-list/2004-December/253517.html(tfloattsumR(R ((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pytmean3sicC sšddlm}t|ƒt|ƒks.t‚|dks@t‚d}x:tt|ƒƒD]&}|t||||ƒ|7}qYW||dt|ƒƒS(så Calculates the minkowski distance between two points. PARAMETERS x - the first point y - the second point p - the order of the minkowski algorithm. Default = 2. This is equal to the euclidian distance. If the order is 1, it is equal to the manhatten distance. The higher the order, the closer it converges to the Chebyshev distance, which has p=infinity iÿÿÿÿ(tpowiigð?(tmathRRtAssertionErrortrangetabsR(txtytpRRti((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pytminkowski_distance9s$c C sÇg}d}x´|D]¬}g}d}x€|D]x} |dk rZ||krZ|j|ƒn@|r„||kr„|j|||ƒn|j||| ƒƒ|d7}q,W|j|ƒ|d7}qW|S(sÇ Takes a list and generates a 2D-matrix using the supplied combination function to calculate the values. PARAMETERS list - the list of items combinfunc - the function that is used to calculate teh value in a cell. It has to cope with two arguments. symmetric - Whether it will be a symmetric matrix along the diagonal. For example, it the list contains integers, and the combination function is abs(x-y), then the matrix will be symmetric. Default: False diagonal - The value to be put into the diagonal. For some functions, the diagonal will stay constant. An example could be the function "x-y". Then each diagonal cell will be "0". If this value is set to None, then the diagonal will be calculated. Default: None iiN(tNonetappend( tlistt combinfunct symmetrictdiagonaltmatrixt row_indextitemtrowt col_indextitem2((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt genmatrixOs   cC sd}t|dƒ}x:|D]2}x)|D]!}ttt|ƒƒ|ƒ}q*WqWd|}d||}x|D]}|t|ƒGHqrWdS(s† Prints out a 2-dimensional list cleanly. This is useful for debugging. PARAMETERS list - the 2D-list to display is %%%is |t|N(Rtmaxtstrttuple(RtmaxlentcolcounttcoltcelltformatR$((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt printmatrixxs   #  cC s?ddlm}d}x|D]}||d7}qW||ƒS(s#calculates the magnitude of a vecoriÿÿÿÿ(tsqrtii(RR2(taR2Rtcoord((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt magnitudeŒs  cC s[t|ƒt|ƒkst‚d}x0tt|ƒƒD]}|||||7}q7W|S(s,Calculates the dotproduct between two vecorsi(RRR(R3tbtoutR((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt dotproduct”s cC s[g}xHtt|dƒƒD]0}|j|g|D]}||^q3ƒƒqWt|ƒS(s/returns the central vector of a list of vectorsi(RRRR+(RtmethodR7RR((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pytcentroidœs.tClustercB sneZdZd„Zd„Zd„Zd„Zd d„Zd„Z d„Z dd „Z d „Z d „Z RS( s  A collection of items. This is internally used to detect clustered items in the data so we could distinguish other collection types (lists, dicts, ...) from the actual clusters. This means that you could also create clusters of lists with this class. cC sd|j|jfS(Ns(t_Cluster__levelt_Cluster__items(tself((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt__str__«scC s |jƒS(N(R?(R>((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt__repr__®scG s:||_t|ƒdkr'g|_nt|ƒ|_dS(sX Constructor PARAMETERS level - The level of this cluster. This is used in hierarchical clustering to retrieve a specific set of clusters. The higher the level, the smaller the count of clusters returned. The level depends on the difference function used. *args - every additional argument passed following the level value will get added as item to the cluster. You could also pass a list as second parameter to initialise the cluster with that list as content iN(R<RR=R(R>tleveltargs((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt__init__±s  cC s|jj|ƒdS(ss Appends a new item to the cluster PARAMETERS item - The item that is to be appended N(R=R(R>R#((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRÃscC s |dkr|jS||_dS(sÉ Sets or gets the items of the cluster PARAMETERS newItems (optional) - if set, the items of the cluster will be replaced with that argument. N(RR=(R>tnewItems((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pytitemsÌs cG s{g}t|ƒdkr$|j}n|djƒ}x@|D]8}t|tƒrf||j|ƒ}q;|j|ƒq;W|S(s8 Completely flattens out this cluster and returns a one-dimensional list containing the cluster's items. This is useful in cases where some items of the cluster are clusters in their own right and you only want the items. PARAMETERS *args - only used for recursion. i(RR=REt isinstanceR;t fullyflattenR(R>RBtflattened_itemst collectionR#((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRG×s   cC s|jS(s< Returns the level associated with this cluster (R<(R>((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRAísicC s`|dd|jGHxE|jD]:}t|tƒrG|j|dƒq|dd|GHqWdS(s= Pretty-prints this cluster. Useful for debuging s s [level %s]is%sN(R<R=RFR;tdisplay(R>tdepthR#((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRJós cC sl|jd}|jd}t|tƒr8|jƒ}n|}t|tƒr\|jƒ}n|}||fS(së Returns the structure (topology) of the cluster as tuples. Output from cl.data: [])>, ])>])>])>])>])>])>] Corresponding output from cl.topo(): ('CVS', ('34.xls', (('0.txt', ('ChangeLog', 'ChangeLog.txt')), ('20060730.py', ('.cvsignore', ('About.py', ('.idlerc', '.pylint.d'))))))) ii(R=RFR;ttopology(R>tlefttrighttfirsttsecond((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRLþs  cC s™|jd}|jd}|jƒ|kr9|jƒgSt|tƒr|jƒ|krt|tƒrƒ|jƒg|j|ƒS|jƒg|ggSndt|tƒr|jƒ|krt|tƒrç|j|ƒ|jƒgS|gg|jƒgSnt|tƒr9t|tƒr9|j|ƒ|j|ƒSt|tƒr_|j|ƒ|ggSt|tƒr…|gg|j|ƒS|g|ggSdS(só Retrieve all clusters up to a specific level threshold. This level-threshold represents the maximum distance between two clusters. So the lower you set this threshold, the more clusters you will receive and the higher you set it, you will receive less but bigger clusters. PARAMETERS threshold - The level threshold NOTE It is debatable whether the value passed into this method should really be as strongly linked to the real cluster-levels as it is right now. The end-user will not know the range of this value unless s/he first inspects the top-level cluster. So instead you might argue that a value ranging from 0 to 1 might be a more useful approach. iiN(R=RARGRFR;tgetlevel(R>t thresholdRMRN((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRQs&   !!N(RRt__doc__R?R@RCRRRERGRARJRLRQ(((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyR;£s      tBaseClusterMethodcB sJeZdZd„Zd„Zd„ZeeƒZd„ZeeƒZ RS(s1 The base class of all clustering methods. cC s ||_||_||_dS(sL Constructs the object and starts clustering PARAMETERS input - a list of objects distance_function - a function returning the distance - or opposite of similarity ( distance = -similarity ) - of two items from the input. In other words, the closer the two items are related, the smaller this value needs to be. With 0 meaning they are exactly the same. NOTES The distance function should always return the absolute distance between two given items of the list. Say, distance(input[1], input[4]) = distance(input[4], input[1]) This is very important for the clustering algorithm to work! Naturally, the data returned by the distance function MUST be a comparable datatype, so you can perform arithmetic comparisons on them (< or >)! The simplest examples would be floats or ints. But as long as they are comparable, it's ok. N(tdistancet_inputt_data(R>tinputtdistance_function((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRCUs  cC s|jdjƒS(sm Returns the structure (topology) of the cluster. See Cluster.topology() for information. i(tdataRL(R>((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyttoporscC s|jS(s< Returns the data that is currently in process. (RW(R>((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt __get_datazscC s|jS(sB Returns the raw data (data without being clustered). (RV(R>((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt__get_raw_datas( RRRSRCR[t_BaseClusterMethod__get_datatpropertyRZt _BaseClusterMethod__get_raw_datatraw_data(((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRTPs     tHierarchicalClusteringcB sbeZdZdd„Zd„Zd„Zd„Zd„Zd„Zd d d d„Z d „Z RS( sõ Implementation of the hierarchical clustering method as explained in http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/hierarchical.html USAGE >>> from cluster import HierarchicalClustering >>> # or: from cluster import * >>> cl = HierarchicalClustering([123,334,345,242,234,1,3], lambda x,y: float(abs(x-y))) >>> cl.getlevel(90) [[345, 334], [234, 242], [123], [3, 1]] Note that all of the returned clusters are more that 90 apart tsinglecC s-tj|||ƒ|j|ƒt|_dS(sQ Constructor See BaseClusterMethod.__init__ for more details. N(RTRCtsetLinkageMethodtFalset'_HierarchicalClustering__clusterCreated(R>RZRYtlinkage((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRC˜s cC sy|dkr|j|_nZ|dkr6|j|_n?|dkrQ|j|_n$|dkrl|j|_n td‚dS(sà Sets the method to determine the distance between two clusters. PARAMETERS: method - The name of the method to use. It must be one of 'single', 'complete', 'average' or 'uclus' RctcompletetaveragetuclussAdistance method must be one of single, complete, average of uclusN(tsingleLinkageDistanceRgtcompleteLinkageDistancetaverageLinkageDistancet uclusDistancet ValueError(R>R9((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRd¤s    cC s™t|tƒs|g}n |jƒ}t|tƒsB|g}n |jƒ}g}x8|D]0}x'|D]}|j|j||ƒƒqhWq[Wt|ƒS(s< The method to determine the distance between one cluster an another item/cluster. The distance equals to the *average* (median) distance from any member of one cluster to any member of the other cluster. PARAMETERS x - first cluster/item y - second cluster/item (RFR;RGRRUR (R>RRt distancestktl((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRn·s       !cC s™t|tƒs|g}n |jƒ}t|tƒsB|g}n |jƒ}g}x8|D]0}x'|D]}|j|j||ƒƒqhWq[Wt|ƒS(s: The method to determine the distance between one cluster an another item/cluster. The distance equals to the *average* (mean) distance from any member of one cluster to any member of the other cluster. PARAMETERS x - first cluster/item y - second cluster/item (RFR;RGRRUR(R>RRRpRqRr((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRmÏs       !cC s©t|tƒs|g}n |jƒ}t|tƒsB|g}n |jƒ}|j|d|dƒ}x:|D]2}x)|D]!}t||j||ƒƒ}q|WqoW|S(s3 The method to determine the distance between one cluster an another item/cluster. The distance equals to the *longest* distance from any member of one cluster to any member of the other cluster. PARAMETERS x - first cluster/item y - second cluster/item i(RFR;RGRUR)(R>RRtmaxdistRqRr((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRlçs       #cC s©t|tƒs|g}n |jƒ}t|tƒsB|g}n |jƒ}|j|d|dƒ}x:|D]2}x)|D]!}t||j||ƒƒ}q|WqoW|S(s4 The method to determine the distance between one cluster an another item/cluster. The distance equals to the *shortest* distance from any member of one cluster to any member of the other cluster. PARAMETERS x - first cluster/item y - second cluster/item i(RFR;RGRUtmin(R>RRtmindistRqRr((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRks       #c C s|dkr!d}d}g}nxlt|ƒdksB|gkrt|j|jtdƒ}d}d}d}xr|D]j}d}xQ|D]I} ||krÈ| |ks³|dkrÈ||f}| }n|d7}q‰W|d7}qvW|d7}||d|d}t||j|d|j|dƒ} |jj|jt|d|dƒƒ|jj|jt |d|dƒƒ|jj | ƒq$Wt|_ dS(s´ Perform hierarchical clustering. This method is automatically called by the constructor so you should not need to call it explicitly. PARAMETERS matrix - The 2D list that is currently under processing. The matrix contains the distances of each item with each other level - The current level of clustering sequence - The sequence number of the clustering iiiN( RRR'RWRgtTrueR;tremoveR)RtRRf( R>R!RAtsequencet smallestpairt mindistancetrowindexR$t cellindexR/tcluster((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyR}s2  !  $   ( (( cC sFt|jƒdkr|jS|js2|jƒn|jdj|ƒS(sç Returns all clusters with a maximum distance of in between each other PARAMETERS threshold - the maximum distance between clusters SEE-ALSO Cluster.getlevel(threshold) ii(RRVRfR}RWRQ(R>RR((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRQVs  N( RRRSRCRdRnRmRlRkRR}RQ(((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRbˆs     9tKMeansClusteringcB s>eZdZdd„Zd„Zd„Zd„Zd„ZRS(s; Implementation of the kmeans clustering method as explained in http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/kmeans.html USAGE ===== >>> from cluster import KMeansClustering >>> cl = KMeansClustering([(1,1), (2,1), (5,3), ...]) >>> clusters = cl.getclusters(2) cC sò||_||_t|ƒ|_t|ƒdkr•t|dtƒr•t|dƒ}x<|dD]-}t|ƒ|kratd|dƒ‚qaqaWnt|ƒdkrÖt|dtƒ rÖ|dkrÖtdƒ‚n|dkrît|_ndS(sx Constructor PARAMETERS data - A list of tuples or integers. distance - A function determining the distance between two items. Default: It assumes the tuples contain numeric values and appiles a generalised form of the euclidian-distance algorithm on them. iisHEach item in the data list must have the same amount of dimensions. Itemswas out of line!sMYou supplied non-standard items but no distance function! We cannot continue!N( t_KMeansClustering__dataRURt!_KMeansClustering__initial_lengthRFRRoRR(R>RZRUtcontrol_lengthR#((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRCws  %2 cC s |dkrtd|ƒ‚n|jgksRt|jƒdksR||jkrY|jS||jkr„td|j|fƒ‚n|j|j|ƒt}xb|tkrt}xI|jD]>}x5|D]-}|j||ƒ}|tkrÉ|}qÉqÉWq¼Wq W|jS(sš Generates clusters PARAMETERS n - The amount of clusters that should be generated. n must be greater than 1 isLWhen clustering, you need to ask for at least two clusters! You asked for %dshUnable to generate more clusters than items available. You supplied %d items, and asked for %d clusters.( RRRR€tinitialiseClustersRvRet_KMeansClustering__clusterst assign_item(R>R t items_movedR}R#tres((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyt getclusters“s$ 3  cC s~|}xJ|jD]?}|j|t|ƒƒ|j|t|ƒƒkr|}qqW||krv|j|||ƒtStSdS(s´ Assigns an item from a given cluster to the closest located cluster PARAMETERS item - the item to be moved origin - the originating cluster N(RƒRUR:t move_itemRvRe(R>R#torigintclosest_clusterR}((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyR„·s0  cC s#|j|j|j|ƒƒƒdS(sÖ Moves an item from one cluster to anoter cluster PARAMETERS item - the item to be moved origin - the originating cluster destination - the target cluster N(Rtpoptindex(R>R#R‰t destination((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyRˆÊs cC smg|_x$t|ƒD]}|jjgƒqWd}x0|D](}|j||j|ƒ|d7}q=WdS(sê Initialises the clusters by distributing the items from the data evenly across n clusters PARAMETERS input - the data set (a list of tuples) clustercount - the amount of clusters (n) iiN(RƒtxrangeR(R>RXt clustercountRtcountR#((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyR‚Ös  N( RRRSRRCR‡R„RˆR‚(((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyR~js    $  N((((RSt __future__RttypesRt ExceptionRRR RRReRR'R1R5R8R:R;RTRbR~(((s</public2/ftp/pub/ccp4/6.5/ccp4-6.5/lib/py2/libtbx/cluster.pyts   )    ­8â