Quilt:一个数据包管理器,像管理代码那样管理数据
Quilt:一个数据包管理器,像管理代码那样管理数据。采用Python编写。Package Composition
kevinemoore released this
Compiler:
- Allow building packages out of other packages and elements from other packages. A new build-file keyword,
package
inserts a package (or sub-package) as an element in the package being built.
Catalog:
- Upgrade router and other dependencies
- Display packages by author
Quilt Teams
kevinemoore released this
Catalog Changes to support private registries
- Amin UI for controlling users and access
- Auditing views
Globbing for package builds
- Allow specifying sets of input files in build.yml
Command-line support for private registries
- Specify teams packages
- Admin commands to create and activate/deactivate users
Better Progress Bar
kevinemoore released this
Version 2.9.1 introduces a better progress bar for installing (downloading) Quilt packages. Quilt push now sends objects' uncompressed size to the registry. The progress bar is now based on the total bytes downloaded instead of the number of files.
Shared Local Package Storage
kevinemoore released this
Shared Local Package Storage
Import packages from shared local directories to save storage overhead and network traffic when sharing packages on the same local network.
Registry Install Stats
Log package installs in the registry to display stats on package use.
New Python API commands
- generate
- rm
- search
Drop support for Python 3.4
(BETA) Team Registries
Updates to commands and local storage to allow users to connect to different registries to support teams running private registries for internal sharing.
Fix download retry
kevinemoore released this
Fixes a bug in download that prevented retrying failed downloads.
Remove Unneeded Pandas dependency
kevinemoore released this
Hotfix for several quilt commands
dimaryaz released this
PR #290
Add Quilt Catalog
kevinemoore released this
Quilt Catalog
Source for the Quilt data catalog is now included in this repository.
MySQL->Postgres
Ported the Quilt registry from MySQL to Postgres
Docker Compose
Improvements to the docker configuration that allows running the registry, catalog, database and authentication service from Docker compose.
Parallel Download
Data fragments can now be downloaded in parallel leading to much faster package installs for large packages.
Centralized Local Package Store
kevinemoore released this
Release Highlights
Quilt packages live in a centralized location on your machine
Quilt data packages are now available wherever you run Python. We recommend that users quilt push all local packages to the registry before upgrading. Further details on migration are here.
Faster builds with build cache
Quilt now caches build intermediates. So if you wish to update the README of a multi-gigabyte package, you can rebuild the entire package in one second.
Group-level build parameters
You can now specify build parameters (like transform) for all children of a group in one shot. The updated syntax and docs are here.
quilt.yml is like requirments.txt but for data
You can now express dependencies on multiple packages in a single file. Docs here.
Experimental: build a package from a GitHub repo
Quilt build now accepts GitHub URLs. If you use data stored on GitHub you can turn it into a Quilt package with quilt build.
Version 2.7.1 includes several minor bug fixes and one new feature, checks. Checks allow a user to specify data integrity checks that are enforced during quilt build.
Downloads
- Support installing subpackages as
quilt install usr/pkg/path
- Upload fragments in parallel
- Use http sessions when accessing S3
Downloads
2.6.3 bump version number to take session fix
Downloads
This release adds a new command to delete a package including all versions and history from the registry.
Downloads
Building a package from a directory of input files now skips generating a build file. That speeds up the build process and makes it easier to change the package contents and rebuild.
Downloads
Watchers:25 |
Star:490 |
Fork:24 |
创建时间: 2017-02-10 10:46:03 |
最后Commits: 昨天 |
许可协议:Apache-2.0 |
Metadata only package install
Assets
Compiler
Registry