Each quarter, the EMC Isilon Uptime Bulletin describes some of the best practices to help you achieve the highest level of operational reliability for your Isilon environment. The latest issue of the Uptime Bulletin covers the following topics:
Benefits of the Isilon OneFS Job Engine The Job Engine is the OneFS work distribution system. Learn how the Job Engine can make routine cluster maintenance easy and efficient.
Best practices for working with snapshots in OneFS Snapshots are used to protect data against accidental deletion. Learn best practices that can help improve snapshot management in OneFS.
Recommended releases of OneFS Get links to the release notes for the latest recommended release versions of OneFS 7.0, 6.5, and 6.0.
The latest firmware package releases Review the following firmware package release information:
Supported drives in drive firmware package 1.6
Supported storage platforms in node firmware package 9.0.1
The EMC Isilon Uptime Bulletin is published at the end of each quarter. If you have any suggestions or requests for the next issue, email firstname.lastname@example.org.
In the EMC Isilon OneFS® operating system, the Job Engine manages the scheduling, reporting, and the initiation of cluster protection and maintenance tasks. A significant benefit of the Job Engine is that it can adaptively manage maintenance jobs based on the amount of cluster resources available.
To ensure that your cluster is performing at optimal levels, follow these Job Engine best practices:
Schedule jobs to run during your cluster’s low usage hours.
When possible, use the default priority, impact, and scheduling settings for each job.
Ensure that the cluster, including individual node pools, is less than 90 percent full.
If the EMC Isilon SmartPools™ software module for resource management is licensed, ensure that default spillover setting is enabled.
Set up notification rules to alert you when the cluster begins to reach capacity thresholds and enable virtual hot spares (VHS) to reserve free space that’s needed in case you need to rebuild data if a disk failure occurs.
Do not disable the Snapshot Delete job.
Run the FSAnalyze job only if you are using InsightIQ. FSAnalyze creates data for the InsightIQ file system analytics tools and provide details about data properties and space usage with the OneFS file system.
We recently learned that our blog readers are most interested in the new EMC Isilon SmartDedupe software offering and the Job Engine enhancements incorporated into in the recently released EMC Isilon OneFS 7.1.
In this video, we take a closer look at these features. First, we cover basic concepts about SmartDedupe and Job Engine performance enhancements in OneFS 7.1. Next, we provide brief demonstrations of how to use these features in the OneFS web administration interface. This video also highlights details about data at rest encryption in OneFS 7.1.
Hello, I’m André Morrissen, a Senior Technical Writer with the Information Development team.
Version 7.1 of OneFS contains numerous enhancements that will improve the performance of your Isilon cluster.
In this video, we’ll take a look at a SmartDedupe, job engine improvements, and data at rest encryption and find out how they can improve your workflow.
SmartDedupe is a new licensed feature of OneFS which enables you to save storage space on your cluster by reducing redundant data—in other words, by deduplicating that data.
SmartDedupe is most beneficial for workflows that incorporate large amounts of duplicate data, such as archiving or when a large amount of virtual machines are stored on a cluster.
As you write files to the cluster, some of those files or blocks of data in the files might be duplicates. You can run a deduplication job that scans the file system to see if that data already exists. If it does, OneFS moves that data to a hidden file called a shadow store and replaces the duplicate data in the files with a pointer to the shadow store.
Deduplication is applied at the subdirectory level and targets all files and directories underneath one or more root directories.
The deduplication job is set to run at low priority by default, so impact to your workflow should be minimal. However, it’s a good idea to wait until users have finished modifying their files on the cluster before you run the job.
You can perform the following deduplication tasks from the OneFS web administration interface.
Assign specific subdirectories for deduplication.
Run an assessment job to determine how much space you might save in a given directory.
And view detailed reports of deduplication jobs.
OneFS 7.1 includes major improvements to the job engine, the system that helps you schedule and manage maintenance jobs on your cluster.
As with previous versions of OneFS, the job engine can adjust jobs based on the amount of cluster resources available. For example, if clients require more system resources, threads allocated to the job engine are decreased.
However, now you can run up to three jobs simultaneously, with a few exceptions that keep similar types of jobs from colliding. For example, you can run an AutoBalance, IntegrityScan, and DedupeAssessment job all at the same time.
OneFS 7.1 also introduces support for Data at Rest Encryption. With this feature, you’ll be able to create a cluster of nodes that contain self-encrypting drives or SEDs. Data at Rest Encryption provides data security that meets specific regulatory requirements for financial and governmental workflows.
Isilon’s use of hardware-based encryption provides the following benefits:
Less consumption of system resources.
Removed drives remain encrypted, which prevents data theft.
The data encryption is performed at the drive level using special processors on each SED that provide 256-bit AES encryption protection. The encryption has less than 1% impact on the performance of the drives themselves.
If you’re interested in creating a cluster with SEDs and using Data at Rest Encryption, contact your account representative.
For more information about the features in this videos, see the OneFS Web Administration.
For a full list of new features, see the OneFS 7.1 Release Notes.
Scanning the file system to ensure that data is protected.
Removing duplicate data.
These are examples of cluster maintenance tasks that are managed by the Job Engine, a work distribution service in the EMC Isilon OneFS operating system.
The Job Engine plays an important role in maintaining the reliability and integrity of your cluster, and is well documented on the EMC Online Support site. It’s also a popular feature with customers, as indicated by our recent poll results.
This post provides a brief overview of the Job Engine and a video demonstration of how to start a job.
Job Engine basics
The Job Engine prioritizes, schedules, and balances the amount of resources dedicated to a specific set of tasks, or jobs, that run in the background. Specifically, the Job Engine distributes job segments—broken into phases and tasks—across the nodes in your Isilon cluster. For a list of cluster maintenance jobs that are managed by the Job Engine, see the OneFS administration guides or the knowledgebase article titled OneFS 5.0 – 7.0: Complete list of jobs by OneFS version. (Log in to the EMC Online Support site to view this article and OneFS administration guides.)
You can review the schedule of jobs and monitor the progress of running jobs in the OneFS web administration interface or command-line interface. Because some jobs can take many hours to complete or consume a significant amount of resources while running, you can also manually start, pause, or stop jobs. If a job is paused, you can restart the job at any time from the point at which it was interrupted.
In OneFS versions 6.0.x through 7.0.x, only one job can run at a time in the background. In OneFS 7.1, up to three jobs can run at the same time (with a few exceptions to keep similar types of jobs from interfering with each other). Each job has a default impact policy and priority to determine when it will run. You can also modify these default settings when you start a job.
How to start a job
If you want to start a job before it is scheduled to run automatically, you can start it manually. This basic procedure for OneFS 6.0.x through 7.0.x is described in the following video.
To start a job in OneFS 7.1, go to Cluster Management > Job Operations > Job Types click More (circled in red below), and then click Start Job.
The opinions and interests expressed on Dell EMC employee blogs are the employees' own and do not necessarily represent Dell EMC's positions, strategies or views. Dell EMC makes no representation or warranties about employee blogs or the accuracy or reliability of such blogs. When you access employee blogs, even though they may contain the Dell EMC logo and content regarding Dell EMC products and services, employee blogs are independent of Dell EMC and Dell EMC does not control their content or operation. In addition, a link to a blog does not mean that EMC endorses that blog or has responsibility for its content or use.