New Feature: Open Extract

Open Extract is a new feature in BIQ, it provides true open access to Python-based analysis. It leverages both the BIQ Engine and the Viewer to allow the user direct access to dataset data from Python programs.

Open Extract can access transactional data, summary OLAP data, reference sets, descriptive statistics, and both transactional and nodal computed measures (CML) for any of the millions of nodes contained in a dataset.


Open Extract Performance

Direct access to the BIQ Engine data allows Open Extract scripts to perform tasks that are difficult or impossible for traditional OLAP engines. The scripts run surprisingly fast, with the ability to scan over 1 million rows of in-memory data per second.


Open Extract IDE

The embedded Python IDE (integrated development environment) provides syntax and error highlighting.

Open Extract integrates with existing Python installations (Versions 3.5 or newer) allowing use of advanced scientific analysis packages such as NumPy, SciPy, SciKit, and Anaconda.


New Open Extract Reports

BIQ v5.04 ships with built-in Open Extract reports including this vendor/commodity segmentation report, which shows the spend breakdown by commodity for the top N vendors. We will continue to add new built-in reports that leverage the unique capabilities of Open Extract.


Chopper Enhancements

The Chopper is now able to combine several input files into a single output file. Specify the order and a header setting for each file and run it through an existing transform script.

We've added macros to Chopper expressions, allowing the user to emit a value for the current line number or the current input file name.


Other Improvements

Usability:

  • Every dropdown for selecting a dimension or measure name now supports filtering and sorting.
  • The phrase-mapping interface now supports filtering on results.
  • Users can now right-click on a dimension block in the Viewer to edit that dimension.
  • Users can now include dimension node name at any level in transaction extracts.
  • Version 5.04 now offers context-sensitive button labels.
  • Many configuration and extract windows have been de-cluttered.


Performance:

  • The Viewer now refreshes and repaints much faster.
  • The phrase-mapper initialization and operation performance has been dramatically improved.
  • Speed for datasets containing a large number of meta rollup dimensions has been increased.
  • Transaction file handling performance has been improved.