PythonCK package

Subpackages

• PythonCK.decorators package
  • Submodules
  • PythonCK.decorators.cache_to_file module
  • PythonCK.decorators.decorators module
  • PythonCK.decorators.lazy module
  • PythonCK.decorators.meta module
  • PythonCK.decorators.safe_makedir module
  • PythonCK.decorators.shelves module
  • Module contents
• PythonCK.logging package
  • Submodules
  • PythonCK.logging.filters module
  • PythonCK.logging.handlers module
  • PythonCK.logging.logger module
  • PythonCK.logging.utils module
  • Module contents

Submodules

PythonCK.ioutils module

Collections of tools related to I/O.

PythonCK.ioutils.capture(*args, **kwds)

REF: http://stackoverflow.com/questions/5136611/capture-stdout-from-a-script-in-python

Usage:

>>> with capture() as std_out_err:
...   print('hi')
>>> out,err = std_out_err
>>> out
'hi\n'
>>> err
''

PythonCK.ioutils.checksum(filepath)

Given filepath to specific file, return its checksum (unique identifier). Optimized for large file, buffered reading.

Parameters:filepath (str) – String to filepath Returns:hex string checksum

>>> checksum('tests/res/ioutils/checksum.txt')
'220e9a9970406e4c688e2c27b8858073f6e2bd33'

PythonCK.ioutils.dump_to_file(obj, destination=None, suffix='')

Quickly dump file, creating folder/file along the way. For instance, where python/xml option file is needed to be created on-the-fly, in order to supply the path to that file for another process.

Parameters:

• obj (object) – Arbitary object to write. The str(obj) will be called. If it’s string and looks like URL, it’ll download instead.
• destination (str) – String for the destination location. If None, create temp file at system’s temp dir. File name will be made from SHA-1 of given object, this is useful for caching to reused dumped object.
• suffix (str) – Suffix (extension) to append to outputfile. Needed in some case where suffix will be looked ahead.

Returns:

String of path to dumped file.

>>> tmpdir = getfixture('tmpdir')
>>> oldcwd = tmpdir.chdir()

>>> dump_to_file('some_data').split('/')[-1]
'256be736caed19be589e439b0d5b8392340d82bc'

>>> dump_to_file('some_data', suffix='.py').split('/')[-1]
'256be736caed19be589e439b0d5b8392340d82bc.py'

>>> dump_to_file('some_data', 'target').replace(str(tmpdir), '...')
'.../target'

>>> dump_to_file('some_data', 'target', suffix='.py').replace(str(tmpdir), '...')
'.../target.py'

>>> dump_to_file('some_data', 'target.py', suffix='.py').replace(str(tmpdir), '...')
'.../target.py'

>>> dump_to_file('some_data', 'dname/target.py').replace(str(tmpdir), '...')
'.../dname/target.py'

>>> _ = oldcwd.chdir()

PythonCK.ioutils.humansize(nbytes)

Given the value in bytes, return the human-readible string.

Parameters:nbytes (int) – Size in BYTES Returns:Human-readible size in string. Return type:str

Usage:

>>> humansize(None) is None
True
>>> humansize(0)
'0 B'
>>> humansize(12)
'12.00 B'
>>> humansize(12000)
'11.72 KB'
>>> humansize(1E10)
'9.31 GB'

PythonCK.ioutils.size(path, force_reload=False, early_giveup=False)

Retrive the total size of given path to directory. Try to be smart by caching the directory size, and invalidate result by checking the st_mtime of that path.

Parameters:

• path (str) – Path to directory to check
• force_reload (bool) – If True, the new result will be calculated regardless the cache.
• early_giveup (bool) – If True, will return 0 immediately is this request has no previous result cached.

Returns:

int representing size in BYTES

Caveats:

• Because disc usage != file size, use ls instead of du (compatibility) with OSX’s BSD’s du

Usage:

>>> import time
>>> tmpdir = getfixture('chtmpdir')

## In case of non-existent path, return zero.
>>> size('non/existent/path')
0

## New content
>>> tmpdir.join("hello.txt").write("content")
>>> size(tmpdir)
7

## Second file in the same directory, invalidate the cache.
>>> time.sleep(1)  # Need at least 1 second for st_mtime to propagate
>>> tmpdir.join("hello2.txt").write("contents".encode('utf-8'))
>>> size(tmpdir)
15

## Make sure that if directory disappear, the cache will be deleted.
>>> tmpdir.mkdir('sub').join('hello3.txt').write('content')
>>> size(tmpdir.join('sub'))
7
>>> tmpdir.join('sub').remove()
>>> size(tmpdir.join('sub'))
0

## DEV: early_giveup, force_reload
>>> tmpdir.mkdir('sub2').join('test.txt').write('foobar')
>>> size(tmpdir.join('sub2'), early_giveup=True)
0
>>> size(tmpdir.join('sub2'), force_reload=True)
6

REF: http://superuser.com/questions/22460/how-do-i-get-the-size-of-a-linux-or-mac-os-x-directory-from-the-command-line

PythonCK.itertools module

Collections of useful iteration-related tools, just like default itertools

class PythonCK.itertools.EnumContainer

Bases: tuple

Constructor for enumeration-pattern, subclassing tuple

Usages:

>>> TauTypes = EnumContainer('e', 'h1', 'h3', 'mu', 'other')
>>> TauTypes
('e', 'h1', 'h3', 'mu', 'other')
>>> isinstance( TauTypes, tuple )
True

>>> mytautype = TauTypes('h1')    # Constructor style init
>>> mytautype = TauTypes.h1       # Attribute style init
>>> TauTypes('BAD')
Traceback (most recent call last):
  ...
AttributeError: No element named: "BAD" in this enum.

>>> mytautype
'h1'
>>> mytautype == 'h1'   # String-based check
True
>>> mytautype.h1        # Attr-based check
True
>>> mytautype == 'h3'   # This check should return False
False
>>> mytautype == 'UNKNOWN'
False
>>> mytautype + 'h1' # concat
'h1h1'
>>> TauTypes.index(mytautype)  # Get int-index
1

__call__(...) <==> x(...)

__getattr__(s)

__module__ = 'PythonCK.itertools'

static __new__(cls, *args)

__slots__ = ()

class PythonCK.itertools.EnumElement

Bases: str

Base clss for enumeration object

__getattr__(s)

__module__ = 'PythonCK.itertools'

__slots__ = ()

PythonCK.itertools.chunks(l, n)

Yield successive n-sized chunks from l.

Parameters:

• l (iterable) – Itrable instance to be splitted into chunks .
• n (int) – Len of each chunk.

Usage:

>>> result = chunks( range(10), 3 )
>>> result
<generator object chunks at ...>
>>> list(result)
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

REF:

• http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python

PythonCK.itertools.flatten(l)

Better handling with generator for arbitary-depth list flattening. (In exchange for speed penalty).

https://stackoverflow.com/questions/2158395/flatten-an-irregular-list-of-lists-in-python

Parameters:l (iterable) – Iterable instance to be flatten.

Usage:

>>> flatten( [1, 2, (3, 4), [5]] )
<generator object flatten at ...>

>>> list(flatten( [1, 2, (3, 4), [5]] ))
[1, 2, 3, 4, 5]

>>> list(flatten(123)) # not iterable, return simply list of that object
[123]

PythonCK.itertools.merge_dicts(*dicts)

Merge the dictionary recursively

Used prominently in combining the recipe in gaudi_simulation.py

Usage:

>>> d = merge_dicts( {1:1}, {2:2} )
>>> sorted(d.items())
[(1, 1), (2, 2)]

>>> merge_dicts( {1:1}, {1:2} )
Traceback (most recent call last):
  ...
Exception: Conflict ...

>>> d = merge_dicts( {1: {100:100}}, {1: {200:200}} )
>>> d == { 1: {100:100, 200:200} }
True

>>> merge_dicts( {1:1}, {1:1} )
{1: 1}
>>> merge_dicts( {1:[1]}, {1:[2]} )
{1: [1, 2]}

PythonCK.itertools.powerset(iterable)

Return the powerset of given iterable.

Usage:

>>> result = powerset([1, 2, 3])
>>> result
<itertools.chain object at ...>
>>> list(result)
[(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)]

PythonCK.stringutils module

Utilities related to string

PythonCK.stringutils.concat_multilines(*msgs)

Concat horizontally list of multi-line strings.

Useful for printing long-thin paragraph together.

Parameters:msgs (list of str) – list of paragraphs.

Usage:

>>> print(concat_multilines('line11\nline12', 'line21\nline22'))
line11 line21
line12 line22

PythonCK.stringutils.insert_line(msg, index, line)

Similar to list.insert, for inserting line into a paragraph.

>>> print(insert_line('line1\nline2\nline3', 1, 'xxx'))
line1
xxx
line2
line3

PythonCK.stringutils.lreplace(string, pattern, sub)

Replaces ‘pattern’ in ‘string’ with ‘sub’ if ‘pattern’ STARTS WITH ‘string’. It’s actually better if I remember this regex by heart.

>>> lreplace('prefix_word', 'prefix_', 'new_')
'new_word'

PythonCK.stringutils.remove_vertical_duplicate(paragraph, *tags)

Generic extension where arbitary amount of tags can be used to remove.

>>> print(remove_vertical_duplicate(
...   "muon PT\n"
...   "muon Eta\n"
...   "muon Phi\n"
...   "tau  PT\n"
...   "tau  Eta\n"
...   "tau  Phi"
... , 'muon ', 'tau '))
muon PT
     Eta
     Phi
tau  PT
     Eta
     Phi

PythonCK.stringutils.try_int_else_float(s)

Given the string of number, return the float of that number, and cast to int if possible. Raise exception otherwise

>>> try_int_else_float('2.00')
2
>>> try_int_else_float('2.50')
2.5

Note: python2,3 has slightly different error msg for below.:

>>> try_int_else_float('not_a_number')
Traceback (most recent call last):
...
ValueError: could not convert string to float: ...

Module contents