As our education marketplace grew, we faced a dilemma: we had little visibility into our data.
Questions such as "who is our best customer?", "which target countries are gaining in sales?" or "what are our most popular categories for paid leads?" required tinkering in the Rails console or writing custom stats pages. We lost a lot of developer time over the years.
So we started looking for the best Business Intelligence solution out there.
Unfortunately, they all had their problems. Some required knowledge of SQL. Some made multi-table data exploration a pain. Most cost an arm, a leg and an iPhone. Per month.
The best tool we found was Looker, but at a monthly price equal to our burn rate, it wouldn't work.
So I decided to re-implement the essential parts of Looker as an open source alternative.
Insights is a desktop or self-hosted "SQL-not-required" data analytics and business intelligence tool. Featuring linkable URLs, easy data exploration, automatic joins, graphs, exports, facets (pivots), saveable views, pretty colors and a ridiculously permissive license (MIT).
It's a work in progress and you're brave for checking it out! Cheers!
To install, make sure you have Node 7.6+ installed and then run:
# desktop (electron) version npm install -g insights insights-core insights-desktop insights desktop # run the desktop/electron version # server version npm install -g insights insights-core insights-server insights createsuperuser # create an admin user to access the server insights createsecret # create a new authentication secret at ~/.insights/secret insights server # run the server
Read the deployment guide if you wish to run this on a fresh Ubuntu 16.04 instance.
How does it work
Similar to Looker and their LookML, Insights requires a mapping of your database's structure to function.
This mapping is either autogenerated from your schema and foreign keys, or created with scripts such as the
insights_export gem, which exports Rails (ActiveRecord) associations.
Once you have a mapping, you may then edit it to add custom columns (e.g.
full_name: first_name || ' ' || last_name), hide existing fields (e.g.
encrypted_password) or hide entire models.
(Note: as of this moment, it's only possible to edit the mapping if you generate the .yml file manually. The editor for the autogenerated structures is coming soon!)
How to use
Play with the demo here and try to answer the following questions:
- Which product has been bought the most? (solution)
- Sales by country by month (solution) - export it as a PDF as well!
- Sales by delivery status (solution)
- Where are your users from (solution)
- ... by month? (solution)
- ... only ones with confirmed orders? (solution)
Hint: to count rows, select the
id field and then
count from the table header.
- Structure editor in the interface
- Manage users in the interface
- PDF and XLSX exports (they worked on the old Rails version, but have to be ported to NodeJS)
- Better graph controls
- Graphs that don't require a time column
- View generated SQL
- Dashboard item grouping/tabbing
- Multiple lines from different sources on one chart in the dashboard
insights is built as a monorepo and uses
oao for package management, at least until yarn workspaces become a thing.
oao uses yarn internally, so you need to have it installed as well.
If you want to help with development, run these steps:
# 1. fork the repo in github # 2. clone it git clone firstname.lastname@example.org:<YOUR_NAME_HERE>/insights.git # 3. install all dependencies cd insights yarn # 4. run the development environment yarn start # 5. open http://localhost:3030/ and hack away
How to publish new versions
Here's the latest workflow to publish a new version. This may change in the future.
# install oao npm install -g oao # optional: check if the core builds as needed. yarn run build:core # update the versions of the repos under packages/* and the root repo oao all "npm version patch" npm version patch # sync package.jsons oao prepublish # publish oao all "npm publish"