Week starting 07/15/19 (Week 7)

-

This week I started clustering the data after using multiple distance algorithms on the v, specifically Dynamic Time Warping (DTW), Fourier Coefficients, Autocorrelation, and the Euclidean Distance. After the clusters have been calculated, a cluster validity algorithm have to be run in order to determine the optimal number of clusters. The cluster validity algorithm included the Average Silhouette Width (ASW), the Calinski Harabasz Score (CHS), and the Davies-Bouldin Score (DBS). These metrics are set up in such a way that we want to obtain the highest Average Silhouette Width (ASW) and Calinski Harabasz Score (CHS) and the lowest Davies-Bouldin Score (DBS). For the variable air temperature, the best results so far have come from running DTW and running that through a dimensionality reduction algorithm called Uniform Manifold Approximation and Projection (UMAP).

Result of the silhouette width where the red line is the average (Left) and visual representation of data with three clusters


Results of the cluster validity metrics, where the red lines show the most promising number of clusters.

Average of each cluster plotted daily, where the parabolic shape contains sites in the northern hemisphere and the sites in the southern hemisphere are contained in the two inverse parabolas.

Comments

Post a Comment

Popular posts from this blog

Week 5

-
Image
We gave presentations on our progress so far and wrote a paper about the materials and the methods that will be used for the rest of this internship. I was able to get the half-hourly data to be split into sets of 48 elements using the split function. Once I had done that, I calculated euclidean distances between each of these sets of 48 to get 365 distances which was then placed into a vector to be plotted. The comparisons so far were with AT_Neu_NEE_f 2002, 2003, 2004. The 2002 dataset was missing the first 16 days worth of data, therefore those vectors that compared with 2002 had less elements and when plotted had to be shifted as to not show any data for the first 16 days. We will continue these calculations with other variables and use other methods to compare the datasets.
Read more

week 4 weekend

-
Image
This weekend we went to lands end in san francisco. We were on the trail for no more than 3 minutes when Alex, Will and I decided to go over the ropes and wonder off trail. Alex and I went on an adventure and were scaling down the rocks and getting closer to the water. It was a lot of fun until we realized how far down we were and had to go back up. However we got to take a lot of amazing pictures! When we came back up we lasted another 5 minutes before wondering off trail again on an adventure. This time we ended up getting out socks wet, but again it allowed us to get great pictures. This adventure was through a small cave. When it came out we were in this area where the water would hit and it would fly up when it hit the rocks like in the little mermaid (excellent movie). After this side adventure we got back on the trail and walked for a while, we came to an area where we could go in the water a bit and have our feet in the sand. Will and I stood back where the water would hit our ...
Read more

Week 3 (06/17/19 - 06/21/19)

-
The algorithm that was going to be developed to analyze the FluxNet dataset has changed from Locality-Sensitive Hashing to now using statistical machine learning to identify patterns in the dataset and track changes in it. I finished the research proposal.
Read more