Utilities - unihan_etl.util#

Utilities for parsing UNIHAN’s data and structures.

unihan_etl.util.ucn_to_unicode(ucn)[source]#

Return a python unicode value from a UCN.

Converts a Unicode Universal Character Number (e.g. "U+4E00" or "4E00") to Python unicode (u'\u4e00')

Return type:

str

Parameters:

ucn (str) – 

>>> ucn_to_unicode("U+4E00")
'\u4e00'

>>> ucn_to_unicode("4E00")
'\u4e00'
unihan_etl.util.ucnstring_to_python(ucn_string)[source]#

Return string with Unicode UCN (e.g. “U+4E00”) to native Python Unicode (u’u4e00’).

Return type:

bytes

Parameters:

ucn_string (str) – 

>>> ucnstring_to_python("U+4E00")
b'一'
unihan_etl.util.ucnstring_to_unicode(ucn_string)[source]#

Return ucnstring as Unicode.

Return type:

str

Parameters:

ucn_string (str) – 

>>> ucnstring_to_unicode('U+4E00')
'一'

>>> ucnstring_to_unicode('U+4E01')
'丁'

>>> ucnstring_to_unicode('U+0030')
'0'

>>> ucnstring_to_unicode('U+0031')
'1'
unihan_etl.util._dl_progress(count, block_size, total_size, out=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)[source]#

MIT License: https://github.com/okfn/dpm-old/blob/master/dpm/util.py

Modification for testing: http://stackoverflow.com/a/4220278

Return type:

None

Parameters:

  • count (int) –

  • block_size (int) –

  • total_size (int) –

  • out (IO[str]) – 

>>> _dl_progress(0, 1, 10)
Total size: 10b

>>> _dl_progress(0, 100, 942_200)
Total size: 942Kb
unihan_etl.util.merge_dict(d, u)[source]#

Return updated dict.

Return type:

TypeVar(T, bound= Mapping[str, t.Any])

Parameters:
Returns:

Updated dictionary

Return type:

dict

Notes

Thanks: http://stackoverflow.com/a/3233356

unihan_etl.util.get_fields(d)[source]#

Return list of fields from dict of {filename: [‘field’, ‘field1’]}.

Return type:

List[str]

Parameters:

d (UntypedUnihanData) –