SMQTK v0.7.0 Release Notes¶
This minor release incorporates various fixes and enhancements to representation and algorithms interfaces and implementations.
A new docker image has been added to wrap the IQR web interface and headless services. This image can either be used as a push-button image ingestion and IQR interface container, or as a fully feature environment to play around with SMQTK, Caffe deep-learning-based content description and IQR.
A major departure has happened for some representation structures, like DataElements, as they are no longer considered hashable and now have interfaces reflecting their mutability. Representation structures, by their nature of having arbitrary backends, may be modifiable my external agents interacting in a separate manner with the backend being used. This has also opened up the ability to provide algorithm implementations with DataElement instances instead of filepaths for desired byte content and many implementations have transitioned over to using this pattern. There is nothing fundamentally wrong with requesting file-path input, however it is restricting as to where configuration files or data models may come from.
Updates / New Features since v0.6.2¶
- Added KWCNN DescriptorGenerator plugin
setup.pyscript in support of installation by
pip. Updated CMake code to install python components via this scripts.
SMQTK_BUILD_LIBSVMto CMake for optionally building libSVM and Flann (both default ON).
- Added default
ClassificationElementFactorythat uses the in-memory back-end.
- Added minibatch kmeans based descriptor clustering function with CLI interface.
- Revised implementation of in-memory representation, doing away with global cache.
- Added optimization to Postgres backend for a slightly faster
- Removed lingering assumption of
pyflannmodule presence in
- Added initial Ansible roles for SMQTK and Caffe dependency.
- Revised default IQR service configuration file to take into account recently added session expiration support. Defaults were used before, but now it needs to be specifically enabled as by default expiration is not enabled.
- Added IQR / playground docker container setup. Includes: - CPU + NVIDIA GPU capable docker file. - Optional input image tiling. - Optional startup of RESTfule NN and IQR services.
- Updated build and installation documentation.
- Added missing utility script documentation hooks.
- Standardized utility script definition of argument parser generation function for documentation use.
- Added initial simple Girder plugin to link to an external IQR webapp instance.
- Added algo/rep/iqr imports to top level
__init__.pyto make basic functionality available without special imports.
- Added plugin for Girder-hosted data elements
from_urimember function as well as global function to handle instance construction or selection via URI string specification.
- Postgres data element will now automatically create its configured table if it doesn’t exist and authentication and sufficient privileges.
- Postgres descriptor element will now automatically create its configured table if it doesn’t exist and authentication and sufficient privileges.
- Postgres descriptor index will now automatically create its configured table if it doesn’t exist and authentication and sufficient privileges.
- Add script to conveniently make Ball-tree hash index model given an existing
hash2uuids.picklemodel file required for the
compute_many_descriptor.pybatch size parameter now defaulted to 0 instead of 256.
- Add script to cluster an index of descriptors via mini-batch kmeans (scikit-learn).
- Added script wraping the use of the mini-batch kmeans descriptor clustering function.
- Added scripts and notebooks for retrieving MEMEX-specific data from ElasticSearch.
- Moved-command line scripts to the smqtk.bin sub-module in order to use
setuptoolsupport for cross-platform executable generation.
classifier_kfold_validationutility now only uses MemoryClassificationElement instead of letting it be configurable.
- Added script for finding nearest neighbors of a set of UUIDs given a nearest neighbors index.
- Added script to add GirderDataElements to a data set
- Started a module containing URL-base utility functions, initially adding a url-join function similar in capability to
- Added fixed tile cropping to image transform tool.
- Added utility functions to detect mimetypes of files via
- Updated/Rearchitected IqrSearchApp (now IqrSearchDispatcher) to be able to spawn multiple IQR configurations during runtime in addition to any configured in the input configuration JSON file. This allows external applications to manage configuration storage and generation.
- Added directory for Girder plugins and added an initial one that, given a folder with the correct metadata attached, can initialize an IQR instance based on that configuration, and then link to IQR web interface (uses existing/updated IqrSearch web app).
- Added ability to automatically login via a valid Girder token and parent Girder URL for token/user verification. This primarily allows restricted external IQR instance creation and automatic login from Girder redirects.
- Mongo session information block at bottom IQR app page now only shows up when running server in debug mode.
- Added document showing complete use case with IQR RESTful webservice using the IQR docker image with LEEDS Butterfly data. Includes expected results users should be able to replicate.
Fixes since v0.6.2¶
- Fixed issues caused by moving scripts out of ./bin/ to ./python/smqtk/bin.
- Fix logging bug in compute_many_descriptors.py when file path has unicode in it.
- Removed final loop progress report from compute_many_descriptors.py as it did not report valid statistics.
- Fixed deprecated import of
- Fixed DescriptorFileElement cache-file save location directory when configured to use subdirectories. Now no longer creates directories to store only a single file. Previous file-element roots are not compatible with this change and need to be re-ingested.
- Fixed IQR web app url prefix check
- Fixed cosine distance function to return angular distance.
SmqtkObjectlogger class accessor name changed to not conflict with
flask.Flasklogger instance attribute.
- Fixed Flow upload browse button to not only allow directory selection on Chrome.