Main Page   Namespace List   Class Hierarchy   Compound List   File List   Compound Members   File Members  

compresstest.cpp File Reference

This file contains code to test various lossless compression algorithms on financial time series (e.g., stock close price). More...

#include <stdio.h>
#include "invpacktree_int.h"
#include "packtree_int.h"
#include "support.h"
#include "delta.h"
#include "haar_int.h"
#include "ts_trans_int.h"
#include "line_int.h"
#include "costwidth.h"
#include "yahooTS.h"

Go to the source code of this file.

Functions

int* copy (int *intVec, const size_t N)
 Make a copy of an integer array. More...

bool compare (const int *v1, const int *v2, const size_t N)
 Compare two integer arrays of size N. More...

size_t calcPacketWidth (packdata_list< int > &bestBasisList)
 Calculate the number of bits needed to represent the result of the wavelet packet transform. More...

size_t packet_calc (const int *intVec, const int *copyVec, const int N)
 Wavelet packet calculation. More...

size_t delta_calc (int *intVec, const int *copyVec, const int N)
 Calculate the number of bits needed to represent the data after "delta" compression. More...

size_t wave_calc (int *intVec, const int *copyVec, const int N, liftbase< int *, int > *w)
 Calculate the number of bits needed to represent the data after wavelet compression wavelet compression. More...

int main ()
 Read in a set of equity time series file. More...


Detailed Description

This file contains code to test various lossless compression algorithms on financial time series (e.g., stock close price).

The documentation in this file is formatted for doxygen (see www.doxyeng.org).

Copyright and Use

You may use this source code without limitation and without fee as long as you include: This software was written and is copyrighted by Ian Kaplan, Bear Products International, www.bearcave.com, 2002.

This software is provided "as is", without any warranty or claim as to its usefulness. Anyone who uses this source code uses it at their own risk. Nor is any support provided by Ian Kaplan and Bear Products International.

Please send any bug fixes or suggested source changes to:

     iank@bearcave.com

Author:
Ian Kaplan

Definition in file compresstest.cpp.


Function Documentation

size_t calcPacketWidth ( packdata_list< int > & bestBasisList )
 

Calculate the number of bits needed to represent the result of the wavelet packet transform.

The result of this version of the wavelet packet transform is a list of vectors, where each vector represents a best basis fit for a particular region of the signal.

The result returned here is an obvious lower bound, since in practice a vector would consist of a fixed set of values. Also the length of the vector would have to be included, along with the width the the length values.

Definition at line 104 of file compresstest.cpp.

00105 {
00106   packdata_list<int>::handle h;
00107   size_t totalWidth = 0;
00108 
00109   for (h = bestBasisList.first(); h != 0; h = bestBasisList.next( h )) {
00110     packdata<int> *node = bestBasisList.get_item( h );
00111     const size_t len = node->length();
00112     const int *vec = node->getData();
00113     int nodeWidth = support::vecWidth( vec, len );
00114     totalWidth += nodeWidth;
00115   }
00116 
00117   return totalWidth;
00118 } // calcPacketWidth

bool compare ( const int * v1,
const int * v2,
const size_t N )
 

Compare two integer arrays of size N.

Return true if they are equal, false otherwise.

Definition at line 73 of file compresstest.cpp.

Referenced by delta_calc(), main(), packet_calc(), and wave_calc().

00074 {
00075   bool rslt = true;
00076 
00077   for (size_t i = 0; i < N; i++) 
00078   {
00079     if (v1[i] != v2[i])
00080     {
00081       rslt = false;
00082       break;
00083     }
00084   }
00085 
00086   return rslt;
00087 } // compare

int * copy ( int * intVec,
const size_t N )
 

Make a copy of an integer array.

Note that the function allocates memory, but does not deallocate it.

Definition at line 56 of file compresstest.cpp.

00057 {
00058   int *newVec = new int[ N ];
00059 
00060   for (size_t i = 0; i < N; i++)
00061   {
00062     newVec[i] = intVec[i];
00063   }
00064 
00065   return newVec;
00066 } // copy

size_t delta_calc ( int * intVec,
const int * copyVec,
const int N )
 

Calculate the number of bits needed to represent the data after "delta" compression.

Delta compression stores the difference between value si-1 and si.

Definition at line 186 of file compresstest.cpp.

Referenced by main().

00189 {
00190   delta_trans<int> delta;
00191 
00192   delta.forward( intVec, N );
00193 
00194   const size_t deltaWidth = support::vecWidth( intVec, N );
00195 
00196   delta.inverse( intVec, N);
00197 
00198   bool isEqual = compare( intVec, copyVec, N);
00199   if (! isEqual)
00200     printf("Delta compression inverse failed\n");
00201 
00202   return deltaWidth;
00203 } // delta_calc

int main ( )
 

Read in a set of equity time series file.

Calculate the number of bits needed to represent the data without compression and after wavelet and wavelet packet compression.

Definition at line 247 of file compresstest.cpp.

00248 {
00249   const char *files[] = { "aa",    // Alcoa Aluminium
00250                           "amat",  // Applied Materials
00251                           "ba",    // Boeing
00252                           "cof",   // Capital One
00253                           "ge",    // General Electric
00254                           "ibm",   // IBM Corp.
00255                           "intc",  // Intel
00256                           "mmm",   // 3M
00257                           "mrk",   // Merck
00258                           "wmt",   // Wal-Mart
00259                           0        // The null pointer
00260                         };
00261 
00262   const size_t N = 512;
00263   double realVec[ N ];
00264   int intVec[ N ];
00265 
00266   // an instance of yahooTS with the path to the data directory
00267   const char *dataDirPath = "..\\data\\equities\\";
00268   yahooTS ts( dataDirPath );
00269 
00270   printf("Equity Uncompressed  delta  Haar  line  TS    wavelet packet (line)\n");
00271 
00272 
00273   for (size_t i = 0; files[i] != 0; i++) {
00274     
00275     size_t n = N;
00276     if (! ts.getTS( files[i], realVec, n, yahooTS::Close )) {
00277       break;
00278     }
00279 
00280     if (n != N) {
00281       printf("Error: %d out of %d data elements read\n", n, N );
00282       break;
00283     }
00284     
00285     support::decimalToInt( intVec, realVec, N );
00286     
00287     int *copyVec = copy( intVec, N );
00288     
00289     const size_t beforeWidth = support::vecWidth( intVec, N );
00290     
00291     const size_t deltaWidth = delta_calc( intVec, copyVec, N );
00292     
00293     haar_int haar;
00294     const size_t haarWidth = wave_calc( intVec, copyVec, N, &haar );
00295     
00296     line_int<int *> line;
00297     const size_t lineWidth = wave_calc( intVec, copyVec, N, &line );
00298     
00299     ts_trans_int ts_trans;
00300     const size_t tsTransWidth = wave_calc( intVec, 
00301                                            copyVec, 
00302                                            N, 
00303                                            &ts_trans);
00304     
00305     const size_t packetWidth = packet_calc( intVec, copyVec, N );
00306     
00307     printf("  %4s       %4d    %4d   %4d  %4d  %d      %4d\n", 
00308            files[i], beforeWidth, deltaWidth, haarWidth, lineWidth, tsTransWidth, packetWidth );
00309     
00310   }
00311   return 0;
00312 }

size_t packet_calc ( const int * intVec,
const int * copyVec,
const int N )
 

Wavelet packet calculation.

The function returns the minimal total number of bits needed to represent the data when compressed using the wavelet packet algorithm.

Definition at line 130 of file compresstest.cpp.

00133 {
00134   // The "line" wavelet transform (e.g., line with slope)
00135   line_int<packcontainer_int> line;
00136 
00137   // calculate the wavelet packet tree, using the line wavelet transform
00138   packtree_int tree( intVec, N, &line );
00139 
00140   // get the root of the wavelet packet transform tree
00141   packnode<int> *treeRoot = tree.getRoot();
00142 
00143   // Assign a cost on the basis of bit width
00144   costwidth cost( treeRoot );
00145 
00146   // Calculate the "best basis" function from the tree
00147   tree.bestBasis();
00148 
00149   // Check that the best basis function succeeded.  That is,
00150   // that the best basis function does not include the 
00151   // original data.
00152   
00153   if (! tree.bestBasisOK()) {
00154     printf("Best basis calculation failed\n");
00155   }
00156 
00157   // Get the best basis list.  This will be a list of
00158   // nodes consisting of the best basis set.  This set is
00159   // obtained by traversing the tree, top down, left to
00160   // right.
00161   packdata_list<int> bestBasis = tree.getBestBasisList();
00162 
00163   // Sum the cost values (width for each node in the best basis
00164   // list
00165   size_t width = calcPacketWidth( bestBasis );
00166 
00167   invpacktree_int invtree( bestBasis, &line );
00168 
00169   const int *invRslt = invtree.getData();
00170 
00171   bool isEqual = compare( invRslt, copyVec, N );
00172   if (! isEqual)
00173     printf("Wavelet packet inverse is wrong\n");
00174 
00175   return width;
00176 } // packet_calc

size_t wave_calc ( int * intVec,
const int * copyVec,
const int N,
liftbase< int *, int > * w )
 

Calculate the number of bits needed to represent the data after wavelet compression wavelet compression.

Parameters:
intVec   A pointer to an array of integers
copyVec   A pointer to a copy of the integer array. This is used to verify that the inverse(forward(intVec)) transform yields the original data.
N   the size of intVec and copyVec
w   a pointer to a wavelet class

Definition at line 221 of file compresstest.cpp.

Referenced by main().

00225 {
00226   w->forwardTrans( intVec, N );
00227   
00228   const size_t waveWidth = support::vecWidth( intVec, N );
00229 
00230   w->inverseTrans( intVec, N );
00231 
00232   bool isEqual = compare( intVec, copyVec, N);
00233   if (! isEqual)
00234     printf("Line wavelet inverse is wrong\n");
00235 
00236   return waveWidth;
00237 } // wave_calc


Generated at Tue May 27 21:56:16 2003 for Wavelet compression, determinism and time series forecasting by doxygen1.2.8.1 written by Dimitri van Heesch, © 1997-2001