chc.util.fileutil module

File utilities

All intermediate and final results are saved in xml/json files with fixed names derived from the names of the c files and c functions. The functions in this file parse the xml/json files and return the top functional xml element of these files (xml) or the dictionary (json). The filenames themselves can be retrieved as well.

File-naming schema:

A file x.c in a project is identified by three components:

  • a project path <pp>, which can be either the directory of the file (in case of a single file) or the location of the Makefile (in case of a multi file project);

  • a project name <pn>, which is the x in the case of a single c file, and is user-specified name otherwise;

  • a file path <fp>, which is the path of the directory in which the file resides relative to the project path. In case of a single c file fp is omitted.

  • the file name <x> (without extension)

The user can specify a different directory to store the analysis artifacts, which is designated by <tp> (targetpath).

Given a file x.c with project path pp, project name pn, and file path fp, and specified target path tp, the following artifacts are produced in multiple stages:

  1. A new analysis directory is created: tp/pn.cch with two subdirectories: tp/pn.cch/a and tp/pn.cch/s

  2. x.c is preprocessed with the gcc preprocessor, producing the file pp/fp/x.i. Both pp/fp/x.c and pp/fp/x.i are copied to tp/pn.cch/s/fp.

  3. x.c is parsed by the CodeHawk/CIL parser, producing the following files

    • tp/pn.cch/a/fp/x/x_cdict.xml

    • tp/pn.cch/a/fp/x/x_cfile.xml

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_cfun.xml (for every function fn in x)

3a. In case of multiple c files, the files are linked, resolving dependencies

between globally visible functions, variables, and data structures, producing the files:

  • tp/pn.cch/a/fp/x/x_gxrefs.xml

  • tp/pn.cch/a/globaldefinitions.xml

  • tp/pn.cch/a/target_files.xml

  1. When all c files in the project (that is, indicated by the Makefile) have been parsed the directory tp/pn.cch is saved as a gzipped tar file, tp/pn.cch.tar.gz, to enable redoing the analysis without having to reparse, or sharing the same parsing results with others (this is because the preprocessing is performed against the local environment, including header files, availability of other libraries, etc., and thus may be different for different computers).

  2. Primary proof obligations are generated for x.c, producing the following files:

    • tp/pn.cch/a/fp/x/x_cgl.xml

    • tp/pn.cch/a/fp/x/x_ctxt.xml

    • tp/pn.cch/a/fp/x/x_ixf.xml

    • tp/pn.cch/a/fp/x/x_prd.xml

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_api.xml

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_pod.xml

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_ppo.xml

    • tp/pn.cch/a/fp/x/logfiles/x_primary.chlog

    • tp/pn.cch/a/fp/x/logfiles/x_primary.errorlog

    • tp/pn.cch/a/fp/x/logfiles/x_primary.infolog

  3. Invariants are generated and proof obligations are checked, producing the following files:

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_invs.xml

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_vars.xml

    • tp/pn.cch/a/fp/x/logfiles/x_gencheck.chlog

    • tp/pn.cch/a/fp/x/logfiles/x_gencheck.errorlog

    • tp/pn.cch/a/fp/x/logfiles/x_gencheck.infolog

  4. Supporting proof obligations are generated for dependencies across functions and files, producing

    • tp/pn.cch/a/fp/x/functions/<x_fn>/x_fn_spo.xml

Steps 6 and 7 are repeated until convergence.

exception chc.util.fileutil.CFileNotFoundException(filenames: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCAnalysisResultsNotFoundError(path: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCAnalyzerNotFoundError(location: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCArtifactsNotFoundError(path: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCDirectoryNotFoundError(dirname: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCError(msg: str)[source]

Bases: chc.util.fileutil.CHError

exception chc.util.fileutil.CHCFileNotFoundError(filename: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCGuiNotFoundError(location: Optional[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJSONParseError(filename: str, e: ValueError)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJulietCWENotFoundError(cwe: str, cwes: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJulietScoreFileNotFoundError(cwe: str, test: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJulietScoreKeyNotFoundError(cwe: str, test: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJulietTargetFileCorruptedError(key: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJulietTestNotFoundError(cwe: str, test: str, tests: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCJulietTestSuiteFileNotFoundError(filename: str)[source]

Bases: chc.util.fileutil.CHCFileNotFoundError

exception chc.util.fileutil.CHCJulietTestSuiteNotRegisteredError[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCParserNotFoundError(location: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCProjectNameNotFoundError(group: str, name: str, projects: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCSemanticsNotFoundError(path: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCShortCutNameError(name: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCSingleCFileNotFoundError(filenames: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCSummaryHeaderNotFound(header: str, headers: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCSummaryTestNotFound(header: str, fname: str, fnames: List[str])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCTargetGroupFileNotFoundError(filename: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCTargetGroupNotFoundError(group: str)[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHCXmlParseError(filename: str, errorcode: int, position: Tuple[int, int])[source]

Bases: chc.util.fileutil.CHCError

exception chc.util.fileutil.CHError(msg: str)[source]

Bases: Exception

wrap() → str[source]
chc.util.fileutil.archive_project_summary_results(path: str, projectname: str) → None[source]
chc.util.fileutil.check_analysis_results(path: str) → None[source]

Raises an exception if analysis results are not present.

chc.util.fileutil.check_analyzer() → None[source]
chc.util.fileutil.check_cch_semantics(projectpath: str, projectname: str, deletesemantics: bool = False) → None[source]
chc.util.fileutil.check_cfile(path: str, filename: str) → None[source]
chc.util.fileutil.check_cfile_results(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[str][source]
chc.util.fileutil.check_cfun_results(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[str][source]
chc.util.fileutil.check_gui() → None[source]
chc.util.fileutil.check_juliet_test_summary(cwe: str, test: str) → None[source]
chc.util.fileutil.check_parser() → None[source]
chc.util.fileutil.check_semantics(path: str, deletesemantics: bool = False) → None[source]
chc.util.fileutil.chtime(t: float) → str[source]
chc.util.fileutil.create_backup_file(filename: str) → None[source]
chc.util.fileutil.get_analysis_target_index(group: str) → Dict[Any, Any][source]

Returns the dictionary referred to by the group name.

chc.util.fileutil.get_analysisresults_path(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_api_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_api_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_callgraph_filename(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_candidate_contracts(path: str, cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cchpath(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_cfile_assignment_dictionary_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cfile_assignment_dictionaryname(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_cfile(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_contexttable_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cfile_contexttablename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_dictionary_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cfile_dictionaryname(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_directory(path: str, cfilename: str) → str[source]
chc.util.fileutil.get_cfile_filepath(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_fnpath(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_cfile_fnspath(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_interface_dictionary_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cfile_interface_dictionaryname(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_logfile_name(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, kind: str) → str[source]
chc.util.fileutil.get_cfile_logfiles_directory(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_predicate_dictionary_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cfile_predicate_dictionaryname(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cfile_usr_filename(path: str, cfilename: str) → str[source]
chc.util.fileutil.get_cfile_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cfilenamebase(cfilename: str) → str[source]
chc.util.fileutil.get_cfun_basename(path: str, cfilename: str, fname: str) → str[source]
chc.util.fileutil.get_cfun_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_cfun_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cgc_challenge_path(testname: str, targetname: str) → str[source]
chc.util.fileutil.get_cgc_challenges() → Dict[str, Any][source]
chc.util.fileutil.get_cgc_path() → str[source]
chc.util.fileutil.get_cgc_summaries() → str[source]
chc.util.fileutil.get_cgc_test_targets(t: str) → List[Any][source]
chc.util.fileutil.get_chc_artifacts_path(path: str) → str[source]
chc.util.fileutil.get_contracts(path: str, cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_cxreffile_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → str[source]
chc.util.fileutil.get_cxreffile_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_flattened_juliet_testcases() → Dict[str, List[str]][source]
chc.util.fileutil.get_fn_composite(cfilename: str, fnname: str, ext: str) → str[source]
chc.util.fileutil.get_functional_testpath(testpath: str) → str[source]
chc.util.fileutil.get_functional_tests_path() → str[source]
chc.util.fileutil.get_functionindex_filename(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_global_contract(path: str) → Dict[str, Any][source]
chc.util.fileutil.get_global_declarations_xnode(targetpath: str, projectname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_global_definitions_filename(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_global_dictionary_xnode(targetpath: str, projectname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_global_invs_filename(path: str, cfilename: str, objectname: str) → str[source]
chc.util.fileutil.get_global_xml_contract(path: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_group_name(name: str) → str[source]

Returns the group-name from a short-cut name.

chc.util.fileutil.get_invs_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_invs_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_itc_path() → str[source]
chc.util.fileutil.get_itc_testpath(testname: str) → str[source]
chc.util.fileutil.get_juliet_path() → str[source]
chc.util.fileutil.get_juliet_result_times(cwe: str, test: str) → Tuple[str, str][source]
chc.util.fileutil.get_juliet_scorekey(cwe: str, test: str) → Dict[str, Any][source]
chc.util.fileutil.get_juliet_summaries() → str[source]
chc.util.fileutil.get_juliet_target_file() → Dict[str, Any][source]
chc.util.fileutil.get_juliet_testcases() → Dict[str, Dict[str, List[str]]][source]
chc.util.fileutil.get_juliet_testpath(cwe: str, test: str) → str[source]
chc.util.fileutil.get_juliet_variant_descriptions() → Dict[str, Any][source]
chc.util.fileutil.get_kendra_cpath(cfilename: str) → str[source]
chc.util.fileutil.get_kendra_path() → str[source]
chc.util.fileutil.get_kendra_testpath(testname: str) → str[source]
chc.util.fileutil.get_kendra_testpath_byid(testid: int) → str[source]
chc.util.fileutil.get_libc_summary_test(header: str, functionname: str) → Tuple[str, str][source]
chc.util.fileutil.get_libc_summary_test_list() → Dict[str, Any][source]
chc.util.fileutil.get_libc_summary_test_path() → str[source]
chc.util.fileutil.get_parse_archive(targetpath: str, projectname: str) → str[source]

Returns the full path to the parse archive file.

chc.util.fileutil.get_parse_targzname(projectname: str) → str[source]
chc.util.fileutil.get_parse_tarname(projectname: str) → str[source]
chc.util.fileutil.get_pod_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_pod_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_ppo_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_ppo_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_preserves_memory_functions_filename(path: str) → str[source]
chc.util.fileutil.get_project_logfilename(path: str) → str[source]
chc.util.fileutil.get_project_name(name: str) → str[source]

Returns the project name from a short-cut name.

chc.util.fileutil.get_project_path(name: str) → str[source]
chc.util.fileutil.get_registered_analysis_targets() → Dict[str, Any][source]

Returns a dictionary of group -> (path,project-dictionary).

chc.util.fileutil.get_savedsource_path(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_spo_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_spo_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_src_filename(path: str, cfilename: str) → str[source]
chc.util.fileutil.get_srcfile_lines(path: str, cfilename: str) → List[str][source]
chc.util.fileutil.get_targetfiles_filename(targetpath: str, projectname: str) → str[source]
chc.util.fileutil.get_targetfiles_list(targetpath: str, projectname: str) → List[Tuple[Optional[str], Optional[str]]][source]
chc.util.fileutil.get_targetfiles_xnode(targetpath: str, projectname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_testdata_dict() → Dict[str, Any][source]
chc.util.fileutil.get_vars_filename(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → str[source]
chc.util.fileutil.get_vars_xnode(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.get_workshop_file_data(project: str, wfile: str) → Optional[Dict[str, Any]][source]
chc.util.fileutil.get_workshop_list() → Optional[Dict[str, Any]][source]
chc.util.fileutil.get_workshop_path() → str[source]
chc.util.fileutil.get_xnode(filename: str, rootnode: str, desc: str, show: bool = True) → Optional[xml.etree.ElementTree.Element][source]
chc.util.fileutil.has_analysisresults_path(targetpath: str, projectname: str) → bool[source]
chc.util.fileutil.has_candidate_contracts(path: str, cfilename: str) → bool[source]
chc.util.fileutil.has_contracts(path: str, cfilename: str) → bool[source]
chc.util.fileutil.has_global_contract(path: str) → bool[source]
chc.util.fileutil.has_global_xml_contract(path: str) → bool[source]
chc.util.fileutil.is_shortcut_name(name: str) → bool[source]

Returns true if the name is a valid short-cut name.

chc.util.fileutil.list_test_applications() → str[source]
chc.util.fileutil.load_callgraph(targetpath: str, projectname: str) → Dict[str, Any][source]
chc.util.fileutil.load_functionindex(targetpath: str, projectname: str) → Dict[str, Any][source]
chc.util.fileutil.load_preserves_memory_functions(path: str) → Dict[str, Any][source]
chc.util.fileutil.make_cgc_challenge_path(testname: str, targetname: str) → str[source]
chc.util.fileutil.read_cgc_summary_results(testname: str, targetname: str) → Optional[Dict[str, Any]][source]
chc.util.fileutil.read_juliet_test_summary(cwe: str, test: str) → Optional[Dict[str, Any]][source]
chc.util.fileutil.read_project_summary_results(path: str, projectname: str) → Optional[Dict[str, Dict[str, Dict[str, Dict[str, int]]]]][source]
chc.util.fileutil.read_project_summary_results_history(path: str, projectname: str) → List[Dict[str, Any]][source]
chc.util.fileutil.save_api(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str, xnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.save_callgraph(targetpath: str, projectname: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_candidate_contracts_file(path: str, cfilename: str, cnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.save_cfile_interface_dictionary(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, xnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.save_contracts_file(path: str, cfilename: str, cnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.save_cx_file(path: str, cfilename: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_functionindex(targetpath: str, projectname: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_global_xml_contract(path: str, cnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.save_juliet_test_summary(cwe: str, test: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_pod_file(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str, cnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.save_preserves_memory_functions(path: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_project_summary_results(path: str, projectname: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_project_summary_results_as_xml(path: str, projectname: str, d: Dict[str, Any]) → None[source]
chc.util.fileutil.save_spo_file(targetpath: str, projectname: str, cfilepath: Optional[str], cfilename: str, fnname: str, cnode: xml.etree.ElementTree.Element) → None[source]
chc.util.fileutil.unpack_cchtar_file(projectpath: str, projectname: str, deletesemantics: bool = False) → bool[source]
chc.util.fileutil.unpack_tar_file(path: str, deletesemantics: bool = False) → bool[source]
chc.util.fileutil.xfind_node(p: xml.etree.ElementTree.Element, tag: str, msg: str) → xml.etree.ElementTree.Element[source]
chc.util.fileutil.xget_attr(p: xml.etree.ElementTree.Element, tag: str, msg: str) → str[source]
chc.util.fileutil.xget_int_attr(p: xml.etree.ElementTree.Element, tag: str, msg: str) → int[source]