Wind Data Analysis Tools#

Soundarya Sridhar and Jean-Paul Pinelli - Florida Institute of Technology

Florida Tech (FIT) teams deploy networks of wireless sensors on residential houses during high impact wind events or on full scale wind tunnel models. Each team deploys pressure, temperature and humidity sensors alongside anemometry, which includes different anemometers and a conical scanning infrared lidar. The workflow starts with uploading the data to DesignSafe through authentication tokens created in Tapis. Once on DesignSafe, three Jupyter notebooks process and visualize the instruments’ data for analyses. The notebooks provide a user friendly and interactive environment that can adapt to different datasets. For this project, the notebooks perform quasi static real-time analyses, assess sensor performance, study pressure variations for different wind conditions and data correlation. The user interactivity of these notebooks facilitates an easy adaptation to different datasets with little to no-change in code. The example makes use of the following DesignSafe resources:

Jupyter notebooks on DS Juypterhub

Background#

Citation and Licensing#

Please cite Subramanian et al. (2022), Pinelli et al. (2022), J. Wang et al. (2021) and S. Sridhar et al. (2021) to acknowledge the use of any resources from this use case.
Please cite Rathje et al. (2017) to acknowledge the use of DesignSafe resources.
This software is distributed under the GNU General Public License.

Description#

Quasi-real time Data Upload with Tapis#

The user needs a DesignSafe-CI (DS) account. During deployment, data is uploaded to DS in user defined time interval. Tapis CLI and Python 3 executable enable this feature and must be installed on the local system. The user initiates Tapis before every deployment through Windows PowerShell and Tapis creates a token as described below:

Video Tutorial (Timestamps - 28:01 to 35:04): https://www.youtube.com/watch?v=C2McrpQ8XmI

User Guide

Turn on Windows Power Shell and enter the command tapis auth init –interactive.
Enter designsafe for the tenant name,.
Enter the DesignSafe username and password of the authorized user.
Choose to set up Container registry access and Git server access, or skip this step by pressing the return key.
Create a token using the command tapis auth tokens create. At the end, the following response will appear on the cmd line.

Using Jupyter Notebooks#

To save time and memory, the project uses three different notebooks. For any event, either a field deployment or a wind tunnel experiment, the first notebook inputs metadata (sensor information, data columns, timestamp formats) for the dataset and is ideally used once for every event. It outputs a csv file containing the metadata required to run the second notebook. The second notebook calibrates raw data and organizes them into csv and pickled files. This notebook may be run more than once depending on how often new data is uploaded during the event. With the third notebook, users analyze and visualize the data interactively. This is the most frequently used notebook and is run everytime the data needs to be analyzed. There is no need to execute the notebooks sequentially everytime an analysis is done. The figure below illustrates the possible sequences of analysis:

Adaptation to Different Datasets#

The first notebook is a user interactive guide to input important raw data information. This notebook saves time as the user does not have to read, understand and edit the code to change information regarding sensors, columns and data formats. For example, WSNS deployment during the tropical storm Isaias (8/2/2020) used an old and a new WSNS system. The first notebook documented the significant differences in data storage between the two systems. This accelerates data processing as there is no change required in code and the file generated by the notebook acts as a metadata for the second notebook responsible for data processing. The figure below are snapshots of the output file created by the first notebook describing raw data information from two different systems.

Jupyter Notebooks#

Analyses Notebooks and Examples#

The project goal is to measure pressure variation on non-structural components during strong wind events using the network of wireless sensors. The analysis notebooks on DesignSafe are user interactive with markdowns describing the test. They also provide the users with several options to visualize the data. For example, see the analysis notebook for Isaias (tropical storm on August 1-3, 2020). The markdowns have important information and pictures from the deployment, and instructions for the user to easily access data.

And a menu allows user to select from options and look at specific time windows or test conditions.

Using Plotly for Data Driven Animation Frames#

The project objective is to study high impact wind events on non-structural components of residential houses. After the deployment, Jupyter notebooks process and visualize important data for different purposes, including among others: comparisons to ASCE 7 standard; and, assessment of sensor performance with respect to wind conditions. Plotly can create animation frames to look at a snapshot of data from all sensors in different test conditions or even at different timestamps. A single line of code enabled with the right dataframe can quickly reveal trends in the data, and facilitate troubleshooting of any system errors. The figure below shows an application of plotly for one of the Wall of Wind tests for glass sliding doors. The test model was a mock-up box with flat roof, and full scale glass sliding doors were tested at at 105 mph for different wind directions. At uniform velocity, data for each wind direction was collected for 3 minutes and the program computed pressure coefficient Cp values averaged over that time window. A 2D scatter plot was created with x and z dimensions with each point representing a sensor whose color corresponded to a Cp value on the color scale. A single line of code enables the animation frame, which reveals important information:

px.scatter(dataframe, x=x column, y=y column, color=scatter point values, text=text to be displayed for each point, range_color=color scale range, animation_frame=variable for each animation frame, title = plot title)

Including dimensions and trace lines to the plots can add more clarity.

The exercise below is an illustration of these plotly features:

Requirements:

Access Jupyter Notebook on DesignSafe. Once you have your notebook open and you don’t have plotly dash installed, go ahead and use: !pip install dash==1.14.0 --user

Building the Dataframe: Consider a box of spheres that change their numbers ranging from 1 to 10 every hour. You want to look at how the number changes for 12 hours.

Code

# Importing libraries
import random
import pandas as pd

# Define necessary columns
spheres=[1,2,3,4,5]
x=[6,14,10,6,14]
y=[6,6,10,14,14]
rad=[]
for i in range (0,5): #generates 5 random numbers ranging from 1 to 10
    n = random.randint(1,10)
    rad.append(n)
hour=1
Label = ['1','2','3','4','5']

df=pd.DataFrame(spheres,columns=['Sphere']) #dataframe for first hour
df['x']=x
df['y']=y
df['number']=rad
df['hour']=hour
df['label']=Label
for i in range (0,11): #for loop to repeat for the next 11 hours
    hour=hour+1
    temp=pd.DataFrame(spheres,columns=['Sphere'])
    temp['x']=x
    temp['y']=y
    rad=[]
    for i in range (0,5):
        n = random.randint(1,10)
        rad.append(n)
    temp['number']=rad
    temp['hour']=hour
    temp['label']=Label
    df=df.append(temp)  

print(df)

Matching the right columns to suit the syntax will result in an animation frame and a slider!

import plotly.express as px
import plotly.graph_objects as go

fig = px.scatter(df, x='x',y='y', color='number',text="label", animation_frame='hour',title='Magic Box') #animation frame
fig.update_traces(textposition='top center',mode='markers', marker_line_width=2, marker_size=40)                      
trace1 = go.Scatter(x=[2, 2], y=[2, 18],line=dict(color='black', width=4),showlegend=False) #Tracelines to create the box
trace2 = go.Scatter(x=[2, 18], y=[18, 18],line=dict(color='black', width=4),showlegend=False)
trace3 = go.Scatter(x=[18, 18], y=[18, 2],line=dict(color='black', width=4),showlegend=False)
trace4 = go.Scatter(x=[18, 2], y=[2, 2],line=dict(color='black', width=4),showlegend=False)
fig.add_trace(trace1)
fig.add_trace(trace2)
fig.add_trace(trace3)
fig.add_trace(trace4)
fig.update_layout(autosize=False,width=500,height=500,showlegend=True)
fig.show()