In this contest, the dataset has been divided into three subsets, namely training set, test set and last transaction set. The training set is obtained from the original dataset by removing a specified number of users (called test users) with their accompanying data. In order to make sure the items that have viewed by test users also exist in training set, the items should occur at least 20 times in the training after removing the data of test users. For the removed test users, the last transaction of each of them formed the last transaction set and the remaining data of them formed the test set. You should use the training set to train your model and apply your model on the data in test set to predict the last transaction of test users.

You can download the datasets that have already been generated if you don't want to do it by yourself.
Click here to download the datasets that generated from data of site 5202 (1000 test users, minimum click = 15).
Click here to download the datasets that generated from data of site 9426 (1000 test users, minimum click = 10).
Click here to download the datasets that generated from data of site 3699 (1000 test users, minimum click = 10).
Click here to download the datasets that generated from data of site 9093 (1000 test users, minimum click = 6).
Click here to download the datasets that generated from data of site 8631 (1000 test users, minimum click = 15).
Click here to download the datasets that generated from data of all sites (5000 test users, minimum click = 15).

In GenData class, there are 9 member functions:

• size_t GenData::LoadSiteData(); load all the transaction data of the specified site
• size_t GenData::LoadSiteData(size_t istart, size_t iend); load the transaction data of the specified site with specified range
• void GenData::WriteSiteDataToFile(); write the transaction data of the specified site to a flat file
• void GenData::GenerateTrainTestFiles(size_t numtestusers, size_t minclicks); generate the training set and corresponding test set
• void GenData::SetSiteID (string SiteID); set the current site ID
• string GenData::GetSiteID (); get the current site ID
• void GenData::SetDataFileName (string FileName); set the data file name
• size_t GenData::GetNumTrans (string SiteID); get the number of transactions that have been loaded into the memory
• double GenData::TestHitRate(); test the hit rate of the results

To generate the data, you could do it like this:

1. Use SetDataFileName (string FileName) to set the name of original data file. E.g. SetDataFileName(“DataForClass.txt ”).
2. Use SetSiteID (string SiteID) to set the id of the site from which you would like to build the three subsets. E.g. SetSiteID(“5202”). If you would like to process all the data in the original dataset, simply use SetSiteID(“ALL”).
3. Use LoadSiteData() to load all the data from the specified site. You could also specify the range of the data by LoadSiteData(size_t istart, size_t iend). E.g. LoadSiteData(0, 9999).
4. Use GenerateTrainTestFiles(size_t numtestusers, size_t minclicks) to generate the training set, test set and last transaction set. You could specify the number of test users and the minimum threshold of the clicks, i.e. the users in test set should have viewed minclicks items at least. E.g. GenerateTrainTestFiles(1000, 5). NOTE: the minimum value of minclicks is 2 which means there is one transaction in both test set and last transaction set.
5. The generated files are Train.txt, Test.txt and LastTrans.txt.

Evaluate the Results

An example of how to use GenData class is shown in Contest.cpp

/*========================================================
File name:	Contest.cpp
Author:		Xiwei Wang
Created:	Nov.7, 2010
Last Modified:	Nov.23, 2010

Description:	main file for test purpose
=========================================================*/

#include "GenData.h"

int main()
{	
	GenData *p_gd = new GenData();			// create an object of GenData

	// generate the data files from the original data set
    	p_gd->SetDataFileName("DataForClass.txt");	// set the original data file name
	p_gd->SetSiteID("5202");			// set the site ID
	p_gd->LoadSiteData();				// load all the transactions of site 5202
	p_gd->WriteSiteDataToFile();			// write all the transaction data of site 5202 to the file
	p_gd->GenerateTrainTestFiles(1000, 15);	// generate the three subsets with 1000 users in test set and the minimum view count is 15

    	// test the hit rate for the results
	cout << "Hit Rate:" << p_gd->TestHitRate() << endl;
    	
	delete p_gd;	//delete the object
	return 0;
}

Click here to download contest.tar