Manage your data with the new EMC Isilon SDK

Gwen Zierdt

Gwen Zierdt

Principal Technical Content Developer at EMC
Award-winning technical writer for the IT Pro and web service developer.

The Isilon SDK is now available! The Isilon SDK includes documentation and code samples to help you to develop a customized interface to your Isilon OneFS cluster. The OneFS API is a REST-based HTTP interface that allows automation, orchestration, and provisioning of an Isilon cluster. Using the OneFS API, third-party applications can leverage the capabilities of the OneFS operating system to simplify management, data protection, and provisioning.

Code samples are available through the Isilon community on EMC {code} and from the Isilon GitHub repository. Immediately available for download is the Python Language Bindings for the OneFS API and the Statistics Browser tool.  The Isilon SDK Info Hub connects you to a central source of information and provides additional links to resources and documentation.

To use the Isilon SDK, you need either a physical Isilon cluster, or a OneFS simulator. See the video Technical Demo: EMC Isilon OneFS Simulator on the EMC YouTube channel to learn how to install the OneFS Simulator.

The Isilon SDK is Open Source and available free under the MIT license.

Sign up on the Slack EMC {Code} CodeCommunity and join the #isilon channel. Be part of the conversation about the Isilon SDK and receive news of future updates.

Let us know!

Let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at mailto:isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

The latest Isilon Self-Service Platform is now available

Deborah Eisenstein

Deborah Eisenstein

Principal Technical Content Developer at EMC Isilon Storage Division
Principal Technical Content Developer EMC Isilon Storage Division

If you’ve started using Isilon’s Self-Service Platform, you know how useful its health checks and troubleshooting assistance can be. If you aren’t familiar with the Self-Service Platform, see our previous blog post.

EMC Isilon’s Serviceability team is continually working to improve the tool by adding new capabilities. For example, one new feature in version 1.5.6 checks for outdated patches installed on the cluster. Another new feature tells you whether any EMC Technical Advisories (ETAs) might affect your cluster.

To make sure that you have the most up-to-date version of the tool, visit the Self-Service Platform Info Hub. On the Info Hub, you can download the tool, read the latest documentation, and start a discussion with the development team.

As always, we look forward to hearing from you about your experience.

Let us know!

Let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at mailto:isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Announcing EMC Isilon InsightIQ 4.0

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

We’re on a roll!  EMC Isilon InsightIQ 4.0 is available now. If your mission includes monitoring and analyzing the performance and file systems of Isilon clusters, InsightIQ is your hero.  It provides powerful performance monitoring and reporting tools to help you maximize the performance of your EMC Isilon scale-out storage system. InsightIQ includes innovative, advanced analytics to optimize applications, correlate workflow and network events, and accurately forecast future storage needs.

InsightIQ 4.0 Cool New Features

You can use InsightIQ to check that your cluster is operating at peak performance, compare changes in performance across metrics such as network traffic, client activity, and CPU usage, correlate critical cluster events with performance changes, determine how workflows, software, and systems affect cluster performance over time, create reports to help identify performance bottlenecks or to validate a theory, and more.  And of course, InsightIQ 4.0 is fully compatible with OneFS 8.0.0.0 and IsilonSD Edge, right down to InsightIQ cleverly leveraging the benefits of the redesigned OneFS 8.0.0 File System Analytics job. And InsightIQ 4.0 is now available as a $0 licensed product, which means that you can get community support through the EMC Community Network (ECN) Isilon Community.

But wait, there’s more:

  • You can monitor and predict usage of Isilon storage clusters. Prediction is based on the rate of growth of capacity usage over time and extrapolated to identify when the capacity reaches 90%.
  • You can take advantage of the enhanced dashboard, including at-a-glance infographics that provide current health, status, and operations information for your monitored cluster:
IIQ4 dashboard

InsightIQ 4.0’s enhanced dashboard infographics display current health, status, and operations information. (Click for larger image.)

  • You can easily monitor IPv6 data breakouts. InsightIQ 4.0 reports IPv6 breakouts as separate clients instead of presenting them as one big generic IPv6 client group.
  • You can optionally communicate telemetry data to EMC through EMC Secure Remote Support.

More information

To learn more, check out the InsightIQ Info Hub. There you’ll find a fascinating collection of InsightIQ-related information, including release notes, user and installation guides, videos, and links to EMC Community Network (ECN) blog posts and discussions.

Let us know!

Let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at mailto:isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Announcing IsilonSD Edge

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

Watch out, world: IsilonSD Edge is here! IsilonSD Edge is the software-defined OneFS storage solution you’ve been waiting for. IsilonSD Edge is deployed as a virtual instance of OneFS on a VMware ESXi host running on industry-standard hardware. It runs Isilon OneFS 8.0, so you get the same core functionality as the latest OneFS release while enabling your remote sites to have their own on-site storage solutions. That means that you can extend your data lake from your core data center to your edge locations, including remote and branch offices.  Bonus:  you can download and use a production-ready version of IsilonSD Edge for free! The free license allows you to deploy as many clusters as you’d like for as long as you’d like. You also get access to resources available on the EMC Community Network (ECN) Isilon community.

Go get it!

Download IsilonSD Edge here. While you’re there, check out the cool videos showing IsilonSD Edge in action.

More information

To learn more, check out these resources:

Let us know!

Let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at mailto:isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Announcing OneFS 8.0.0

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

EMC Isilon OneFS 8.0.0 release has some great new and updated features that provide benefits on both the hardware and software sides. For example:

  • You can apply file filtering rules to allow or deny file writers to a specific access zone based on criteria that you specify. You can also apply filtering rules to allow or deny file writes to SMB shares.
  • You can configure overlapping base directories for access zones.
  • You can configure multiple DNS servers for each access zone, also known as multi-tenancy.
  • You can store data in the cloud using CloudPools, enabling you to take advantage of lower-cost cloud storage.
  • The clusterwide event log groups related alerts and enables you to configure a maintenance period to stop alerts from being generated while you’re performing maintenance tasks on the cluster.
  • Simultaneous and rolling upgrades are supported between OneFS 8.0.0 and later releases. When you upgrade from OneFS 8.0.0 and later versions, you can roll back the active upgrade to the earlier version of OneFS. The new upgrade features are available only when you upgrade between OneFS 8.0.0 and later releases.

New Features in OneFS 8.0.0

The following new features are included in OneFS 8.0.0. For details, see the OneFS 8.0.0 Release Notes. You’ll need to log in to EMC Online Support to access release notes.

Authentication

  • You can apply file filtering rules to a specified access zone, including any SMB shares and NFS exports within the access zone.
  • Multi-tenant DNS support enables different tenants on the same Isilon cluster to use different DNS servers to perform hostname lookups. DNS servers are configured by a new network object called a groupnet that’s associated with a single access zone. The DNS cache maintains separate caches for separate groupnets. Groupnets are supported by the SMB, NFS, HDFS, and Swift protocols.
  • You can configure access zones to have a shared base directory, allowing the access zones to share data. Access zones that share a base directory should also share authentication providers.

Cluster configuration

  • You can store your data in the cloud with the CloudPools licensed module. CloudPools supports EMC Isilon, EMC Elastic Cloud Storage Service, EMC ECS Appliance, Amazon S3, and Microsoft Azure as cloud providers. Through file pool policies, you can define which data on your cluster to archive to the cloud. Although file data is moved to remote storage, the files remain visible in OneFS.

Events, alerts, and cluster monitoring

  • The isi statistics commands are now OneFS API-based and support Role-Based Access Control (RBAC). See the OneFS 8.0.0 CLI Administration Guide for details.
  • Here are just some of the many improvements to the clusterwide event log. Be sure to check the OneFS 8.0.0.0 Release Notes for details.
    • Related events are placed in an event group, and those events are managed at the group level – allowing you to resolve or ignore related events with a single action.
    • Alerts are distributed through channels, providing flexibility in determining when notifications should be sent and how, and the distribution list for each notification.
    • You can avoid notification of numerous maintenance events by scheduling maintenance periods. Updating event groups is suspended during the maintenance period and resumes when the period ends.
    • Command-line interface (CLI) commands for the clusterwide event log are now OneFS API-based and support Role-Based Access Control (RBAC). See the OneFS 8.0.0 CLI Administration Guide for details.

File system

  • OneFS 8.0.0 is based on FreeBSD 10, improving CPU usage and overall system performance.

Migration

  • New migration controls enable you to monitor and to pause running NetApp migrations.
  • You can schedule NetApp migrations to run during off-peak hours and make them dormant when network usage is high by specifying a sleep schedule. (This feature is not supported for VNX migrations.
  • Migrating data from NetApp SnapLock volumes is supported in SmartLock Enterprise mode.
  • A single migration ID ties a migration workflow together. You can trace the history of NetApp and VMX migrations by providing the migration IDs.

SMB

  • With SMB continuous availability, you can allow clients to create persistent handles that can be reclaimed after an outage, and specify how long to retain the persistent handle. You can use the continuous availability feature to force strict lockouts on users attempting to open a file that belongs to another handle. This feature is available for servers using Windows 8 or Windows 2012 R2 or higher.
  • You can enable server-side copy so that file data doesn’t have to traverse the network. Avoiding network traversal can improve performance for file copy operations performed by Microsoft Windows clients.

Upgrading to OneFS 8.0.0

All upgrades from earlier releases to OneFS 8.0.0 require a simultaneous reboot of the cluster.

If you want to upgrade to this new release, explore your upgrade options by reviewing the Isilon Supportability and Compatibility Guide. Then, prepare for the upgrade process by reviewing the following documents:

When you’re ready to upgrade, download the OneFS 8.0.0 installation file from the Download section of the EMC Online Support site.

More information

For more technical information about all of the new OneFS 8.0.0 features and enhancements, refer to the OneFS 8.0.0 Documentation Info Hub.

Let us know!

Let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Cool tool: the Isilon Self-Service Platform utility

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

Want to have more control over troubleshooting issues on your Isilon cluster? How about being able to resolve common issues yourself, bypassing the support queue? Well, you’re in luck! The Isilon Self-Service Platform utility (SSP) allows you to do just that. Use the SSP utility to perform first-line troubleshooting on your cluster. You’ll be using the same utility that EMC Isilon Technical Support Engineers and Field Representatives use to prevent and troubleshoot a wide range of known issues that occur on Isilon clusters.
Note: The SSP utility runs only on Windows platforms.

How the SSP utility works

The SSP utility analyzes a cluster log set: it does not run live on the cluster. You first collect the cluster log files (the *.tgz files), then run the SSP utility. The utility runs a series of checks on the cluster log files and generates a diagnostic report of the current health of the cluster. You can run checks for specific categories such as pre-upgrade or health check, or choose specific checks for a custom run.

After the analysis completes, the Overall Health screen presents a summary of the basic health of your cluster. It includes high level details about the cluster and the names and status of the individual nodes on the cluster. Summary results are color-coded: red for critical, orange for needs attention, and green for okay. And to help you resolve issues, the report provides links to the relevant documentation.

You can run the SSP utility using its command line interface or the SSP GUI.

Getting the Self-Service Platform utility

Go to the Isilon Self-Service Platform Info Hub and download and unpack the SSP .zip file. The .zip file contains the SSP utility executable file as well as the user guide.

Collecting the cluster log files

Before running the SSP utility, you must collect the cluster log files. You can use the isi gather command or the OneFS WebUI to collect the files. After they’re collected, copy them to a convenient location that the SSP utility can access.  When you run the SSP utility, you specify the location of the log files, a location for the results files, and choose the check(s) that you want to run. The SSP utility places the generated results files for each test in a separate folder in the results location that you specified.

KB article 304468 explains how to collect the log files, complete with a video demonstration.

Running the SSP utility

To run the SSP utility, double-click the Isilon_Self-Service_Platform.exe file. The main screen appears:

SSP Utility Main Screen
(Click the image to see a larger version.)

Click the Help button in the upper left to view the SSP utility’s FAQ page.

Enter the log location, your service request (SR) number, and the output path for the generated diagnostic report. After you click next, you can choose the checks to run. You’ll see a screen with tabs for each of the check categories: pre-upgrade, post-upgrade, health, firmware, and custom, as the following figure shows.

SSP Utility Pre-Upgrade Checks Screen
(Click the image to see a larger version.)

The checks include:

  • Cluster level checks such as overall health of the cluster, patches installed, InfiniBand configuration and upgrade service status
  • Cluster configuration checks such as whether or not ESRS is enabled, file sharing configuration, priority of all routable gateways and whether any gateways share the same priority
  • Node level checks such as boot flash drive problems, NVRAM battery status, mismatched nodes, node health, amount of free space, and uptime
  • Node status checks such as the DMI log, device errors, kernel open files, NIC status, netstat connections, and the cluster’s var/crash partition
  • Disk level checks such as disk load for all disks in each node, node drive bay health, drive errors, and any errors reported in the idi.log file

You can select the checks to run based on category, such as pre-upgrade check, or use the Custom tab to select only those checks you’re interested in:

SSP utility Select Tests screen
(Click the image to see a larger version.)

After you select the checks you want, click the Run button. You’ll see a progress indicator at the bottom of the screen and the SSP utility’s UI is disabled for the duration of the run.

The report results output structure

After the run completes, the SSP utility presents a summary of the results, similar to the following figure (with identifying information redacted).

SummaryScreen
(Click the image to see a larger version.)

You’ll find the results in the folder you specified for the output. The generated results output file structure is similar to the following.

SSP Utility Results File Structure
(Click the image to see a larger version.)

Go get it!

The SSP utility is a great way to take charge of troubleshooting and performing preventative health checks to resolve or avoid common, known issues on your cluster. Check out the Isilon Self-Service Platform Info Hub for the latest download and documentation. And for information about collecting log files (complete with a video demonstration), see KB article 304468.

Let us know!

Let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

New and popular EMC Isilon technical content for December 2015

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

Check out EMC Isilon’s latest and most popular customer-focused technical content. Each month we highlight newly published technical content developed for Isilon customers, as well as the Isilon Top 10’s:

  • Top 10 Knowledgebase articles
  • Top 10 Customer Support documents
  • Top 10 Isilon Info Hubs

New customer-focused Isilon technical content

Here are links to customer-focused technical content that was published in December. Enjoy!

CONTENT TYPE

LINK AND DESCRIPTION

Cluster Talk Podcast Episode 10: Blinky lights as prank or breakthrough source of data bandwidth, the Hadoop Roundup, solving a bi-directional user mapping problem, highlights from OneFS.NEXT, and why happiness in December was all about the Hamiltons.
Cluster Talk Podcast Podcast: Cluster Talk Hub: Looking for all the Cluster Talk podcasts? You’ll find them here.
Video Cool Commands: Why can’t I access my Windows file?
Video Cool Commands: Why can’t I access my UNIX file?
Video ID.TV video playlist: Looking for every Isilon video ever, including the Cool Commands videos? You’ll find them here on the ID.TV YouTube channel.
Isilon Community (ECN) Blog InsightIQ – Lesser known facts
Isilon Community (ECN) Blog Gotchas of Isilon for VDI file storage & Home Directories
Isilon Community (ECN) Blog SMB throughput benchmarking with fio.exe
Isilon Community OneFS and cluster quorum
Isilon Community (ECN) Blog NFS throughput benchmarks with fio
Info Hub Customer Troubleshooting: How to Add a Drive to a Node
Info Hub Customer Troubleshooting: How to Diagnose and Troubleshoot a Battery Failure
Info Hub Customer Troubleshooting: Troubleshoot a Bit Error Rate (BER) Alert on your Isilon Cluster
Info Hub Customer Troubleshooting: InsightIQ does not Display Data Correctly in the File System Reporting Page
Info Hub Customer Troubleshooting: Troubleshoot Problems with InsightIQ Upgrades
Info Hub Customer Troubleshooting: Troubleshoot Problems with Adding or Importing a Cluster for Monitoring in InsightIQ
Info Hub Customer Troubleshooting: Troubleshoot Problems with Accessing or Logging in to the InsightIQ Web Administration Interface
Info Hub Customer Troubleshooting: InsightIQ does not Display Data Correctly on the Live Performance Monitoring Page
Info Hub Customer Troubleshooting: Troubleshoot Problems with Installing or Setting up InsightIQ
Info Hub Customer Troubleshooting: Troubleshoot Windows Active Directory Authentication
Info Hub Customer Troubleshooting: Troubleshoot Your LDAP Authentication Provider
Info Hub Customer Troubleshooting: Troubleshoot Windows File System Permissions for your Isilon Cluster
Info Hub Customer Troubleshooting: Troubleshoot Identity Mapping
Info Hub Customer Troubleshooting: Troubleshoot Problems with your NIS Authentication Provider
Info Hub Customer Troubleshooting: Troubleshoot your SmartConnect Configuration
Info Hub Current OneFS Patches
Info Hub InsightIQ

Top 10 Knowledgebase Articles

Next up: December’s top 10 most-viewed Knowledgebase articles, including ETAs and ESAs. You’ll need to login to view them.

  1. ETA 209918: Isilon OneFS: Nodes that have run for more than 248.5 consecutive days may restart without warning which may lead to potential data unavailability
  2. Isilon OneFS: Nodes that have run for more than 248.5 consecutive days may restart without warning which may lead to potential data unavailability (209418)
  3. ESA-2015-175 (211190)
  4. ESA-2015-155 (208181)
  5. ESA-2015-148 (211191)
  6. ESA-2015-181 (211192)
  7. UPDATE: ETA 208773: Isilon OneFS: When used in conjunction with Isilon OneFS, specific versions of the BXE 10GigE network interface card may experience data inconsistency (208773)
  8. UPDATE: ETA 202452: Isilon OneFS: Nodes that have run for 497 consecutive days may restart without warning
  9. ESA-2015-164 (210195)
  10. OneFS: How to reset the CELOG database and clear all historical events (16535)

Top 10 customer-focused technical documents

And now, December’s most popular customer-focused technical documents.

  1. Current Isilon Software Releases
  2. Isilon Supportability and Compatibility Guide
  3. Current Isilon OneFS Patches
  4. Insight IQ 3.2 Installation Guide
  5. Isilon OneFS 7.2.1.0 – 7.2.1.1 Release Notes
  6. Isilon Product Availability
  7. OneFS Upgrade Planning and Process Guide
  8. InsightIQ 3.2.2 Release Notes
  9. Isilon OneFS 7.2.0.0 – 7.2.0.5 Release Notes
  10. X410 Installation Guide

Top 10 Isilon Info Hubs

We got December’s most popular Info Hubs right here.

  1. Isilon Uptime
  2. OneFS Upgrades
  3. Hadoop
  4. OneFS 7.2 Documentation
  5. Current OneFS Patches
  6. OneFS API
  7. Cloudera with Isilon
  8. OneFS 7.2.1 Documentation
  9. InsightIQ
  10. Customer Troubleshooting

Tell us what you want to know! Contact us with questions or feedback about this blog at isi.knowledge@emc.com. To provide documentation feedback or to request new content, contact isicontent@emc.com.

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Check out the latest Customer Troubleshooting Guides

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

Happy New Year! To celebrate, we’ve published a host of new EMC Isilon Customer Troubleshooting Guides.  These guides provide step-by-step troubleshooting instructions to help you solve issues that may affect Isilon clusters, or to walk you through the steps needed to gather important data to help EMC Customer Support solve your problem quickly.

This latest set of guides covers topics including Isilon hardware, authentication and permissions, Networking/SmartConnect configuration, and InsightIQ.

Visit our Customer Troubleshooting Info Hub for the latest list of published guides. More guides are coming soon!

The new guides are:

Check out these troubleshooting resources and let us know what you think. Email us at isicontent@emc.com with your feedback. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Now on YouTube: Cool Commands videos

Andre Morrisen

Andre Morrisen

Instructional Video Producer at EMC/Isilon

If you’ve been tuning in to the Cluster Talk audio podcast, you’ll know that there’s a segment each episode called “Cool Commands” where they talk about a OneFS command and its application. As an extension of this, we’ve created a video-based version of Cool Commands that focuses on how to find out general or specific information or complete certain tasks.

The first episode describes how to find basic cluster health and job information using commands like isi status and isi job status. In the video, you’ll see the command syntax and we’ll highlight important parts of the output. You’ll also learn how to interpret the output to identify potential issues that might affect your cluster’s performance. The second and third episodes describe how to troubleshoot file access issues on Windows and UNIX clients, respectively. These episodes include a great analogy for how permissions work and step by step troubleshooting to resolve a file access issue.

In future episodes, we’ll describe how to change permissions on UNIX files and folders using chmod and changing the ownership of UNIX files and folders using chown. All the Cool Commands episodes are published on the Isilon Support playlist on EMC’s YouTube Channel. If you have any questions about the videos or suggestions for future episodes, we’d love to hear from you.

Cool Commands Episodes

Bookmark the Isilon Support playlist and check out the Cool Commands videos, then let us know what you think. If you have feedback for us about this or any other Isilon technical content, email us at isicontent@emc.com. And thank you!

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)

Announcing: Hadoop Distributions and Products Supported by OneFS

Risa Galant

Risa Galant

Principal Technical Writer at EMC Isilon Storage Division

Looking for the latest info about Isilon’s supported Hadoop distributions and products? You’ll find it on the Hadoop Info Hub’s Hadoop Distributions and Products Supported by OneFS page. This information evolves rapidly, so if a particular Hadoop distribution or product isn’t listed, be sure to check back periodically.

Tell us what you want to know! Contact us with questions or feedback about this blog at isi.knowledge@emc.com. To provide documentation feedback or request new content, contact isicontent@emc.com.

Please take a moment to rate this article (1 star for poor; 5 stars for excellent)