Open Extract is a new feature in BIQ, it provides true open access to Python-based analysis. It leverages both the BIQ Engine and the Viewer to allow the user direct access to dataset data from Python programs.
Open Extract can access transactional data, summary OLAP data, reference sets, descriptive statistics, and both transactional and nodal computed measures (CML) for any of the millions of nodes contained in a dataset.
Direct access to the BIQ Engine data allows Open Extract scripts to perform tasks that are difficult or impossible for traditional OLAP engines. The scripts run surprisingly fast, with the ability to scan over 1 million rows of in-memory data per second.
The embedded Python IDE (integrated development environment) provides syntax and error highlighting.
Open Extract integrates with existing Python installations (Versions 3.5 or newer) allowing use of advanced scientific analysis packages such as NumPy, SciPy, SciKit, and Anaconda.
BIQ v5.04 ships with built-in Open Extract reports including this vendor/commodity segmentation report, which shows the spend breakdown by commodity for the top N vendors. We will continue to add new built-in reports that leverage the unique capabilities of Open Extract.
The Chopper is now able to combine several input files into a single output file. Specify the order and a header setting for each file and run it through an existing transform script.
We've added macros to Chopper expressions, allowing the user to emit a value for the current line number or the current input file name.