2012-04-04 by Stefan Urbanek
I'm glad to announce new release of Brewery – stream based data auditing and analysis framework for Python.
There are quite a few updates, to mention the notable ones:
- new
brewery runner with commands run and graph
- new nodes: pretty printer node (for your terminal pleasure), generator
function node
- many CSV updates and fixes
Added several simple how-to
examples, such as:
aggregation of remote CSV, basic audit of a CSV, how to use a generator
function. Feedback and questions are welcome. I'll help you.
Note that there are couple changes that break compatibility, however they can
be updated very easily. I apologize for the inconvenience, but until 1.0 the
changes might happen more frequently. On the other hand, I will try to make
them as painless as possible.
Full listing of news, changes and fixes is below.
Version 0.8
News
- Changed license to MIT
- Created new brewery runner commands: 'run' and 'graph':
- 'brewery run stream.json' will execute the stream
- 'brewery graph stream.json' will generate graphviz data
- Nodes: Added pretty printer node - textual output as a formatted table
- Nodes: Added source node for a generator function
- Nodes: added analytical type to derive field node
- Preliminary implementation of data probes (just concept, API not decided yet
for 100%)
- CSV: added empty_as_null option to read empty strings as Null values
-
Nodes can be configured with node.configure(dictionary, protected). If
'protected' is True, then protected attributes (specified in node info) can
not be set with this method.
-
added node identifier to the node reference doc
-
added create_logger
-
added experimental retype feature (works for CSV only at the moment)
- Mongo Backend - better handling of record iteration
Changes
- CSV: resource is now explicitly named argument in CSV*Node
- CSV: convert fields according to field storage type (instead of all-strings)
- Removed fields getter/setter (now implementation is totally up to stream
subclass)
- AggregateNode: rename
aggregates to measures, added measures as
public node attribute
- moved errors to brewery.common
- removed
field_name(), now str(field) should be used
- use named blogger 'brewery' instead of the global one
- better debug-log labels for nodes (node type identifier + python object ID)
WARNING: Compatibility break:
- depreciate
__node_info__ and use plain node_info instead
Stream.update() now takes nodes and connections as two separate arguments
Fixes
- added SQLSourceNode, added option to keep ifelds instead of dropping them in
FieldMap and FieldMapNode (patch by laurentvasseur @ bitbucket)
- better traceback handling on node failure (now actually the traceback is
displayed)
- return list of field names as string representation of FieldList
- CSV: fixed output of zero numeric value in CSV (was empty string)
Links
- github sources: https://github.com/Stiivi/brewery
- Documentation: http://packages.python.org/brewery/
- Mailing List: http://groups.google.com/group/databrewery/
- Submit issues here: https://github.com/Stiivi/brewery/issues
- IRC channel: #databrewery on irc.freenode.net
If you have any questions, comments, requests, do not hesitate to ask.
2012-03-09 by Stefan Urbanek
Another minor release of Cubes - Light Weight Python OLAP framework is out. Main change is that backend is no longer hard-wired in the Slicer server and can be selected through configuration file.
There were lots of documentation changes, for example the reference was separated from the rest of docs. Hello World! example was added.
The news, changes and fixes are:
New Features
- Started writing StarBrowser - another SQL aggregation browser with different
approach (see code/docs)
Slicer Server:
- added configuration option
modules under [server] to load additional
modules
- added ability to specify backend module
- backend configuration is in [backend] by default, for SQL it stays in [db]
- added server config option for default
prettyprint value (useful for
demontration purposes)
Documentation:
- Changed license to MIT + small addition. Please refer to the LICENSE file.
- Updated documentation - added missing parts, made reference more readable,
moved class and function reference docs from descriptive part to reference
(API) part.
- added backend documentation
- Added "Hello World!" example
Changed Features
- removed default SQL backend from the server
- moved worskpace creation into the backend module
Fixes
- Fixed create_view to handle not materialized properly (thanks to deytao)
- Slicer tool header now contains #!/usr/bin/env python
Links
- github sources: https://github.com/Stiivi/cubes
- Documentation: http://packages.python.org/cubes/
- Mailing List: http://groups.google.com/group/cubes-discuss
- Submit issues here: https://github.com/Stiivi/cubes/issues
- IRC channel: #databrewery on irc.freenode.net
If you have any questions, comments, requests, do not hesitate to ask.
2011-12-05 by Stefan Urbanek
I am glad to announce new minor release of Cubes - Light Weight Python OLAP framework for multidimensional data aggregation and browsing. The news, changes and fixes are:
New Features
- New method: Dimension.attribute_reference: returns full reference to an attribute
- str(cut) will now return constructed string representation of a cut as it can be used by Slicer
Slicer server:
- added /locales to slicer
- added locales key in /model request
- added Access-Control-Allow-Origin for JS/jQuery
Changes
- Allow dimensions in cube to be a list, noy only a dictionary (internally it is ordered dictionary)
- Allow cubes in model to be a list, noy only a dictionary (internally it is ordered dictionary)
Slicer server:
- slicer does not require default cube to be specified: if no cube is in the request then try default from
config or get first from model
Fixes
- Slicer not serves right localization regardless of what localization was used first after server was
launched (changed model localization copy to be deepcopy (as it should be))
- Fixes some remnants that used old Cell.foo based browsing to Browser.foo(cell, ...) only browsing
- fixed model localization issues; once localized, original locale was not available
- Do not try to add locale if not specified. Fixes #11: https://github.com/Stiivi/cubes/issues/11
Tutorials
Added tutorials in tutorials/ with models in tutorials/models/ and data in tutorials/data/:
- Tutorial 1:
- how to build a model programatically
- how to create a model with flat dimensions
- how to aggregate whole cube
- how to drill-down and aggregate through a dimension
- Tutorial 2:
- how to create and use a model file
- mappings
- Tutorial 3:
- how hierarhies work
- drill-down through a hierarchy
- Tutorial 4 (not blogged about it yet):
- how to launch slicer server
Links
- github sources: https://github.com/Stiivi/cubes
- Documentation: http://packages.python.org/cubes/
- Mailing List: http://groups.google.com/group/cubes-discuss
- Submit issues here: https://github.com/Stiivi/cubes/issues
If you have any questions, comments, requests, do not hesitate to ask.
2011-09-29 by Stefan Urbanek
I am happy to announce another release of Cubes - Python OLAP framework for multidimensional data aggregation and browsing.
This release, besides some new features, renames Cuboid to more appropriate Cell. This introduces backward python API incompatibility.
Main source repository has changed to Github https://github.com/Stiivi/cubes
Changes
- Class ‘Cuboid’ was renamed to more correct ‘Cell’. ‘Cuboid’ is a part of cube with subset of dimensions.
- all APIs with ‘cuboid’ in their name/arguments were renamed to use ‘cell’ instead
- Changed initialization of model classes: Model, Cube, Dimension, Hierarchy, Level to be more “pythony”: instead of using initialization dictionary, each attribute is listed as parameter, rest is handled from variable list of key word arguments
- Improved handling of flat and detail-less dimensions (dimensions represented just by one attribute which is also a key)
Model Initialization Defaults:
- If no levels are specified during initialization, then dimension name is considered flat, with single attribute.
- If no hierarchy is specified and levels are specified, then default hierarchy will be created from order of levels
- If no levels are specified, then one level is created, with name
default and dimension will be considered flat
Note: This initialization defaults might be moved into a separate utility function/class that will populate incomplete model (see Issue #8 )
New features
Slicer server:
- changed to handle multiple cubes within model: you have to specify a cube for /aggregate, /facts,… in form: /cube//
- reflect change in configuration: removed
view, added view_prefix and view_suffix, the cube view name will be constructed by concatenating view prefix + cube name + view suffix
- in aggregate drill-down: explicit dimension can be specified with drilldown=dimension:level, such as:
date:month
This change is considered final and therefore we can mark it is as API version 1.
Links:
If you have any questions, comments, requests, do not hesitate to ask.
2011-06-25 by Stefan Urbanek
New small release is out with quite nice addition of documentation. It does not bring too many new features, but contains a refactoring towards better package structure, that breaks some compatibility.
Documentation updates
Framework Changes
- added soft (optional) dependencies on backend libraries. Exception with useful information will be raised when functionality that depends on missing package is used. Example: “Exception: Optional package ‘sqlalchemy’ is not installed. Please install the package from http://www.sqlalchemy.org/ to be able to use: SQL streams. Recommended version is > 0.7”
- field related classes and functions were moved from ‘ds’ module to ‘metadata’ and included in brewery top-level: Field, FieldList, expand_record, collapse_record
- added probes
Depreciated functions
Streams
- new node: DeriveNode - derive new field with callables or string formula (python expression)
- new SelectNode implementation: accepts callables or string with python code
- former SelectNode renamed to FunctionSelectNode
Enjoy!
Links
