Week 2

-
I have read a lot of papers and feel I am getting the hang on what is going on from a theoretical stand point. I have started coding and am attempting to use faiss, nmslib and falconn algorithms to use Locality Sensitive Hashing, LSH on a small dataset. Eventually I want to be able to use these algorithms to analyze a larger dataset from FLUXNET.

Comments

Post a Comment

Popular posts from this blog

Description of project

-
When scientists collect their data there tends to be an overlap in the acquired data. Due to this computer resources and space are wasted for this duplicated data. In this project we will focus on techniques to help scientists track and identify this duplicated data so that it can be dealt with. We will do this by developing MinHash algorithms. Doing this we hope will help scientists find duplicated data and not have to waste computer space and resources.
Read more

Week starting 07/15/19 (Week 7)

-
Image
This week I started clustering the data after using multiple distance algorithms on the v, specifically Dynamic Time Warping (DTW), Fourier Coefficients, Autocorrelation, and the Euclidean Distance. After the clusters have been calculated, a cluster validity algorithm have to be run in order to determine the optimal number of clusters. The cluster validity algorithm included the Average Silhouette Width (ASW), the Calinski Harabasz Score (CHS), and the Davies-Bouldin Score (DBS). These metrics are set up in such a way that we want to obtain the highest Average Silhouette Width (ASW) and Calinski Harabasz Score (CHS) and the lowest Davies-Bouldin Score (DBS). For the variable air temperature, the best results so far have come from running DTW and running that through a dimensionality reduction algorithm called Uniform Manifold Approximation and Projection (UMAP). Result of the silhouette width where the red line is the average (Left) and visual representation of data with t...
Read more

Week Starting 6/24/19 (Week 4)

-
Image
This week we started working with multiple different methods for determine similarities in the FLUXNET dataset. So far I have created graphs that compare several variables such as Net Ecosystem Exchange, Heat flux, and more for both hourly and daily values. I also worked on a presentation of my progress. Below are the graphs hourly and daily Net Ecosystem Exchange of three Italian towers in 2002. I also created graphs for the absolute value of each difference. Below are the differences between the towers with respect to the Net Ecosystem Exchange.
Read more