IDEALEM
Implementation of Dynamic Extensible Adaptive Locally Exchangeable Measures

Scientific Data Management Group, Lawrence Berkeley National Laboratory

  
By checking this box, you acknowledge that the software and material on this web site is a non-commercial, academic, non-profit use only.

"IDEALEM: Implementation of Dynamic Extensible Adaptive Locally Exchangeable Measures (IDEALEM)" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved.
BY DOWNLOADING, INSTALLING, OR USING THE SOFTWARE, YOU ARE INDICATING YOUR ACCEPTANCE OF THE TERMS AND CONDITIONS HEREIN.
Full text of LICENSE
Full text of COPYRIGHT

Before going to download page...

The lab needs to collect the software distribution info.
Please fill in the information below and press Go! button.
Name
Institution
Email
  




Summary of IDEALEM (Implementation of Dynamic Extensible Adaptive Locally Exchangeable Measures)
  • Data is dynamically reduced with a novel pattern searching method based on statistical similarity.
  • Compressed/reduced data has accurate representation for the original data, i.e., decompressed/reconstructed data has the same statistical data distribution as the original data.
  • The method supports event/feature detection directly on the compressed data.

Fig. 1

Fig. 2

Fig. 3
  • Fig. 1 shows micro PMU (Phase Measurement Unit) data from power grid electricity measurement from one of on-site switches at LBNL (in collaboration with Energy Technology Area).
  • In Fig 2, the data compression ratio (original size in bytes / compressed size) in this use case is 95.23 with only one history buffer, and can be achieved using only 64K bytes of memory.
  • Compared to gzip, IDEALEM compressed data size is under 2% of gzip-compressed data size in bytes.
  • Fig. 3 shows another micro PMU data from power grid electricity measurement from one of on-site switches at LBNL (in collaboration with Energy Technology Area), which results the compression ratio of 242.3 with 255 history buffers.
IDEALEM is an implementation of the data reduction and pattern searching algorithm for streaming data based on Locally Exchangeable Measures, U.S. Patent 10,366,078.
Software is available for commercial and non-commercial use.


Related links:

Questions or comments:

Please send any comments or questions for this site to: sdmsupportlbl.gov

Credits:

This program was constructed by Alex Sim, Dongeun Lee, K. John Wu, and Jaesik Choi based on an earlier algorithm. This program is based on the following research papers:

We gratefully acknowledge the collaboration and helpful suggestions from colleagues and friends for this project. In particular, we thank Emma Stewart, Sean Peisert, Chuck McParland, Reinhard Gentz, Mahdi Jamei, Ciaran Roberts, and Sebastian Ainslie .