News

The EDI Data Portal now supports an additional code-generation service. Customized Python code can now be generated for all data tables, along with previously available R, Matlab, SAS and SPSS code. The generated Python code uses the Pandas module to generate Pandas “data frames” and produce some data summaries. The code can then be modified to perform user-specified analysis.

The idea for adding the Python code generation came from Stace Beaulieu  (Northeast U.S. Shelf Long-Term Ecological Research) and Mark Servilla and Duane Costa (EDI) provided sage advice on which of the many candidate Python data structures would be most useful.  John Porter (Virginia Coast Reserve Long-Term Ecological Research) added the Python code generation into the existing web service (http://www.vcrlter.virginia.edu/webservice/PASTAprog) and Duane Costa then modified the EDI Data Portal infrastructure so that code generation links automatically appear at the bottom of the listing for every EDI data package that includes one or more data tables.

Please contact John with any questions regarding the Python, R , SAS and SPSS stylesheets (email: jhp7e@virginia.edu, Dept. of Environmental Sciences, University of Virginia, Virginia Coast Reserve Long-Term Ecological Research).

A new and improved version of the EMLassemblyline has been released on GitHub. Several enhancements have been added, including a website for exploring functions, use cases, and staying up-to-date on project news. Some functions, templates, and arguments have been deprecated in favor of others that better conceptualize the EMLassemblyline process for users and organize the codebase for developers. All changes are backwards compatible with EMLassemblyline 2.4.6 (the most widely used and stable version) and will be maintained until May 1, 2020. Contact Colin (colin.smith@wisc.edu) if you have any questions or encounter any issues.

The Environmental Data Initiative (EDI) technical team is excited to announce that we have completed the migration of PASTA+ and related services to the latest version (18.04 LTS) of the Ubuntu operating system. A total 18 servers were affected, including those in our production, staging, and development environments. This event also brought to bear new versions of the Tomcat and Jetty servlet containers to our Java infrastructure and the use of NGINX as our primary front-end web server. In addition, we are now using LetsEncrypt SSL certificates throughout our public-facing infrastructure.

As part of our change to the use of LetsEncrypt, we are now redirecting all HTTP requests to HTTPS to enhance and enforce network-level security – especially when connections contain user login credentials in the payload. It has been brought to our attention that some users may be using scripts that rely on the HTTP connection. If this is the case, your software should be modified to either directly use the HTTPS protocol in your request or ensure that your software will follow a 301 “redirect” to HTTPS. The “cURL” command, for instance, will follow redirects if provided the “-L” or “–location” command line flag. For example:

curl -i -L -X GET http://pasta.lternet.edu/package/eml

will redirect to “https://pasta.lternet.edu/package/eml“.

Finally, our migration touched a great deal of system-level configuration, which still may not be perfectly tuned. If you experience any oddity or issues in the use of PASTA+ services, please let us know, along with as much detail as you can provide, at your earliest convenience.

The Environmental Data Initiative (EDI) technical team is planning to upgrade all infrastructure servers with the latest Long Term Support version (18.04) of the Ubuntu operating system (OS) during April. This upgrade will bring parity to all servers managed by EDI and will ensure timely OS updates, which bring both new features and security to the systems. EDI will begin this process by first upgrading the development environment, followed next by the staging environment, and then finally the production environment. This sequence allows us to formulate and test a solid upgrade plan, while evaluating re-configured software applications, including the use of OS-based Tomcat and Jetty servlet containers and the Nginx front-end web server. In addition, all SSL certificates will be provisioned by LetsEncrypt, a non-profit certificate authority run by Internet Security Research Group. Our upgraded infrastructure will position EDI to operate a seamless and secure infrastructure well into the future.