Description of project

-
When scientists collect their data there tends to be an overlap in the acquired data. Due to this computer resources and space are wasted for this duplicated data. In this project we will focus on techniques to help scientists track and identify this duplicated data so that it can be dealt with. We will do this by developing MinHash algorithms. Doing this we hope will help scientists find duplicated data and not have to waste computer space and resources.

Comments

Post a Comment

Popular posts from this blog

Week 5

-
Image
We gave presentations on our progress so far and wrote a paper about the materials and the methods that will be used for the rest of this internship. I was able to get the half-hourly data to be split into sets of 48 elements using the split function. Once I had done that, I calculated euclidean distances between each of these sets of 48 to get 365 distances which was then placed into a vector to be plotted. The comparisons so far were with AT_Neu_NEE_f 2002, 2003, 2004. The 2002 dataset was missing the first 16 days worth of data, therefore those vectors that compared with 2002 had less elements and when plotted had to be shifted as to not show any data for the first 16 days. We will continue these calculations with other variables and use other methods to compare the datasets.
Read more

week 4 weekend

-
Image
This weekend we went to lands end in san francisco. We were on the trail for no more than 3 minutes when Alex, Will and I decided to go over the ropes and wonder off trail. Alex and I went on an adventure and were scaling down the rocks and getting closer to the water. It was a lot of fun until we realized how far down we were and had to go back up. However we got to take a lot of amazing pictures! When we came back up we lasted another 5 minutes before wondering off trail again on an adventure. This time we ended up getting out socks wet, but again it allowed us to get great pictures. This adventure was through a small cave. When it came out we were in this area where the water would hit and it would fly up when it hit the rocks like in the little mermaid (excellent movie). After this side adventure we got back on the trail and walked for a while, we came to an area where we could go in the water a bit and have our feet in the sand. Will and I stood back where the water would hit our ...
Read more

Week 3 (06/17/19 - 06/21/19)

-
The algorithm that was going to be developed to analyze the FluxNet dataset has changed from Locality-Sensitive Hashing to now using statistical machine learning to identify patterns in the dataset and track changes in it. I finished the research proposal.
Read more