General Description

-
Our goal is to create algorithms to detect duplicated or unnecessary data in large data sets. This can easily occur when a scientist collects data and when this happens the data becomes more difficult to deal with and wastes the finite resources that the computer has at it's disposal. This can also lead to a slow down among other issues. Once we detect the unnecessary or duplicated data we would like to find a way to dispose of this data.

Comments

Post a Comment

Popular posts from this blog

Week 5

-
Image
We gave presentations on our progress so far and wrote a paper about the materials and the methods that will be used for the rest of this internship. I was able to get the half-hourly data to be split into sets of 48 elements using the split function. Once I had done that, I calculated euclidean distances between each of these sets of 48 to get 365 distances which was then placed into a vector to be plotted. The comparisons so far were with AT_Neu_NEE_f 2002, 2003, 2004. The 2002 dataset was missing the first 16 days worth of data, therefore those vectors that compared with 2002 had less elements and when plotted had to be shifted as to not show any data for the first 16 days. We will continue these calculations with other variables and use other methods to compare the datasets.
Read more

week 4 weekend

-
Image
This weekend we went to lands end in san francisco. We were on the trail for no more than 3 minutes when Alex, Will and I decided to go over the ropes and wonder off trail. Alex and I went on an adventure and were scaling down the rocks and getting closer to the water. It was a lot of fun until we realized how far down we were and had to go back up. However we got to take a lot of amazing pictures! When we came back up we lasted another 5 minutes before wondering off trail again on an adventure. This time we ended up getting out socks wet, but again it allowed us to get great pictures. This adventure was through a small cave. When it came out we were in this area where the water would hit and it would fly up when it hit the rocks like in the little mermaid (excellent movie). After this side adventure we got back on the trail and walked for a while, we came to an area where we could go in the water a bit and have our feet in the sand. Will and I stood back where the water would hit our ...
Read more

Week 3 (06/17/19 - 06/21/19)

-
The algorithm that was going to be developed to analyze the FluxNet dataset has changed from Locality-Sensitive Hashing to now using statistical machine learning to identify patterns in the dataset and track changes in it. I finished the research proposal.
Read more