Tuesday, June 26, 2012

NetApp Aggregate Snapshot - Complete reference

In general you no need to create aggregate snapshot, a schedule is automatically set up to generate new aggregate Snapshot copies periodically. But if you wish to stop taking snapshot automatically
Disable automatic snapshot schedule

aggr options <aggr_name> nosnap on

Delete Aggr snapshot

snap delete -A aggr_name <aggr_snapshot>

To change the snap reservation

snap reserve -A <aggr_name> 0

How to restore data from aggregate snapshot

As per my knowledge and tested in my lab environment snap restore –A option will restore entire volume to the aggregate, if you don’t want to do this, if you would like to restore a single volume/file you can use “aggr copy start command”

Pre-checks:
  • The volume you lost data from is a flexible volume 
  • Identify an aggregate which is empty so it can be used for destination (could be on another controller also) 
  • Make sure the destination aggregate is either equal or larger than source aggregate 
  • /etc/hosts.equiv has entry for the filer you want to copy data to and /etc/hosts has its IP address added, in case of copying on same controller loopback address (127.0.0.1) should be added in /etc/hosts file and local filername should be in hosts.equiv file 
  • Name of aggregate’s snapshot which you want to copy 

Example:

Let’s say the volume we lost data was ‘vol1’, the aggregate which has this volume is ‘aggr_source’, the aggregate’s snapshot which has lost data is ‘hourly.1’ and empty aggregate where we will be storing data to is ‘aggr_destination’


Execution:
  • Restrict the destination aggregate using  ‘aggr restrict aggr_destination’ 
  • Start the aggregate data copy using ‘aggr copy start –s hourly.1 aggr_source aggr_dest’ 
  • Once the copy is completed online the aggregate using ‘aggr online aggr_dest’ 
  • If you have done copy on same controller, system will rename the volume ‘vol1’ of ‘aggr_destination’ to ‘vol1(1)’ 
  • Now export the volume or lun and you have your all lost data available or you can vol copy command to original aggregate. Delete the old one.

Sunday, June 24, 2012

Netapp volume Snapshot: Complete Reference

A Snapshot is a locally retained point-in-time image of data. NetApp Snapshot technology is a feature of the WAFL (Write Anywhere File Layout) storage virtualization technology that is a part of Data ONTAP, the microkernel that ships with every NetApp storage system.

The high performance of the NetApp Snapshot makes it highly scalable. A NetApp Snapshot takes only a few seconds to create—typically less than one second, regardless of the size of the volume or the level of activity on the NetApp storage system. After a Snapshot copy has been created, changes to data objects are reflected in updates to the current version of the objects, as if Snapshot copies did not exist. Meanwhile, the Snapshot version of the data remains completely stable. A NetApp Snapshot incurs no performance overhead; users can comfortably store up to 255 Snapshot copies per WAFL volume, all of which are accessible as read-only and online versions of the data.

NetApp Snapshot does not require any additional license to create, but for restore its require to add the license
Snaprestore XXXXXX   <License you can see in NetApp console>
filer> license add XXXXXXX

To Create Snapshot for particular volume

filer> snap create –V bali snap2

To check list of snapshots on a particular volume

filer> snap list bali
Volume bali
working...

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Jun 24 22:16  snap2
  1% ( 0%)    0% ( 0%)  Jun 24 21:37  test

To restore data from a particular volume, make sure that you have valid snaprestore license added
Example 1:

I have a file good.txt in bali volume, so we need to give the command as mentioned below, make sure that you should not give the same name of file which you would like to restore in the same dir/path otherwise it will be overwritten

filer> snap restore -s snap2 -r /vol/bali/good-restore.txt '/vol/bali/good.txt'
WARNING! This will restore a file from a snapshot into the active filesystem.  If the file already exists in the active filesystem, it will be overwritten with the contents from the snapshot.

Are you sure you want to do this? y

You have selected file /vol/bali/good.txt, snapshot test
It will be restored as /vol/bali/good-restore.txt
Proceed with restore? y
filer>

Example 2

If you know the filename which you would like to restore you can use below commands. For example if you have a file in qtree called good.txt, you need to fire the command like below

filer> snap restore -t file '/vol/bali/qtree/good.txt'

WARNING! This will restore a file from a snapshot into the active filesystem.  If the file already exists in the active filesystem, it will be overwritten with the contents from the snapshot.

Are you sure you want to do this? y

The following snapshots are available for volume bali:

date            name
------------    ---------
Jun 24 22:16    snap2
Jun 24 21:37    test
Which snapshot in volume bali would you like to revert the file from? snap2
You have selected file /vol/bala_test/qtree/good.txt, snapshot snap2
Proceed with restore? y
 Filer>

There are 2 commands which are in association with Snapshots, those are.

By default the snap reserve for a volume is 20%, but always you have options to change the value depends on your requirement

filer> snap reserve bali
Volume bali: current snapshot reserve is 20% or 2097152 k-bytes.
filer> snap reserve bali 10
Volume bali: current snapshot reserve is 10% or 1048576 k-bytes.

Note: - Reserving space depends on volume size, if you don't have enough space reserved for creating snapshots it will take space from volumes but if you don't have enough space to dump data on a volume it will not take space from snapshot (Concept).

Snapshot Schedule

filer> snap sched bali
Volume bali: 0 2 6@8,12,16,20
filer>
snap sched [-A | -V] [<vol-name> [weeks [days [hours[@<list>]]]]]

To be Continued...

NetApp Deduplication

 Netapp supports deduplication where only unique blocks in the flex volume is stored and it creates a small amount of additional metadata in the de-dup process. The NetApp deduplication technology allows duplicate 4KB blocks anywhere in the flexible volume to be deleted and stores a unique one.
The core enabling technology of deduplication is fingerprints. These are unique digital signatures for every 4KB data block in the flexible volume.
 When deduplication runs for the first time on a flexible volume with existing data, it scans the blocks in the flexible volume and creates a fingerprint database, which contains a sorted list of all fingerprints for used blocks in the flexible volume. After the fingerprint file is created, fingerprints are checked for duplicates and if found, first a byte-by-byte comparison of the blocks is done to make sure that the blocks are indeed identical. If they are found to be identical, the block’s pointer is updated to the already existing data block and the duplicate data block is released and inode is updated.
The below link will make you to understand the detailed information about how de-dup works - Source NetApp Community

 https://communities.netapp.com/community/netapp-blogs/drdedupe/blog/2010/04/07/how-netapp-deduplication-works--a-primer
Deduplication refers to the elimination of redundant data in the storage. In the deduplication process, duplicate data is deleted, leaving only one copy of the data to be stored. However, indexing of all data is still retained should that data ever be required. Deduplication is able to reduce the required storage capacity since only the unique data is stored. 


1. Enable dedup (asis) license:

filer> license add <Code for NearStore>
filer> license add <Code for De-dup>
filer> sis on /vol/bali

2. If you have a new flex volume which was just created, follow this step to enable ASIS deduplication

filer> sis on /vol/bali
Deduplication for "/vol/bali" is enabled.
Already existing data could be processed by running "sis start -s /vol/bali”


3. If you have already existing flex volume with data in it, follow this step.

filer> sis start -s /vol/bali

4. If you have already existing flex volume with data in it, follow this step.

filer> vol status bali
Volume          State   Status          Options
bali      online  raid_dp, flex   nosnap=on
                        sis
Containing aggregate: 'aggrSATA'

5. Check de-dupe status

filer> sis status /vol/bali
Path            State   Status      Progress
/vol/demovol    Enabled Idle        Idle for 00:02:12

6. Check the storage space saved due to deduplication
filer> df -s /vol/bali
Filesystem      used    saved   %saved
/vol/bali/   9316052 0       0%

7. If you have to run de-dupe at a later point of time on this volume, use “sis status /vol/bali”

8. Deduplication can be scheduled by using "sis config" with below options
sis config [ [ [ -s schedule ] | [ -m minimum_blocks_shared ] ] <path> ...]
        - Sets up, modifies and retrieves schedule and minimum blocks shared
         value of SIS volumes
9.  If you wish to stop and delete the de-dupe entries on a specific volume, follow the below steps

filer> sis stop /vol/bali
Operation is currently idle: /vol/bali
filer> sis off /vol/bali
SIS for "/vol/bali" is disabled.
filer> priv set advanced
Warning: These advanced commands are potentially dangerous; use
them only when directed to do so by Network Appliance
personnel.
filer*> sis undo /vol/bali
filer*> priv setfiler> sis status /vol/bali
Disabled Undoing 6810 MB Processed
filer> sis status /vol/bali
No status entry found.

100 useful commands for NetApp Beginners (1 to 10)


Netapp beginner guide has details about basic Netapp commands and an example to explain each command.
 

1. How to find Data Ontap version on a filer :
filer> version
NetApp Release 7.3.1: Thu Jan 8 22:29:32 PST 2012
2. How to find "uptime" of a filer?

The uptime command gives system uptime. Also it gives number of NFS, CIFS, FCP, iSCSI operations on a Netapp filer.

filer> uptime
  8:58pm up 23 days, 43 mins, 1354234 NFS ops, 940 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops
filer>
3. How to get the System Configuration information of a NetApp filer: 



filer> sysconfig  
        NetApp Release 7.3.1: Thu Jan  8 01:31:42 PST 2009
        System ID: 0135025121 (whmn7027)
        System Serial Number: 3093075 (whmn7027)
        System Rev: D1
        System Storage Configuration: Multi-Path
        slot 0: System Board
                Processors:         1
                Memory Size:        896 MB
        slot 0: Dual 10/100/1000 Ethernet Controller G20
                e0a MAC Address:   00:a0:98:0a:85:54 (auto-1000t-fd-up)
                e0b MAC Address:   00:a0:98:0a:85:55 (auto-1000t-fd-up)
        slot 0: FC Host Adapter 0a
                14 Disks:            2959.2GB
                1 shelf with AT-FCX
        slot 0: FC Host Adapter 0b
                14 Disks:            2959.2GB
                1 shelf with AT-FCX
        slot 0: SAS Host Adapter 0c
                1 shelf with ESAS
        slot 0: NetApp ATA/IDE Adapter 0e (0x000001f0)
                0e.0                 249MB
        Baseboard Management Controller:
                Firmware Version:   1.1
                IPMI version:       2.0
                DHCP:               off
                BMC MAC address:    xx:xx:xx:xx:xx:xx
                IP address:         xx.xx.xx.xx
                IP mask:            xx.xx.xx.xx
                Gateway IP address: xx.xx.xx.xx
                BMC ARP interval:   10 seconds
                BMC has (1) user:   naroot
                ASUP enabled:       on
                ASUP mailhost:      whml0242
                ASUP from:          bali@bali.caom
                ASUP recipients:  bali@bali.caom,autosupport@netapp.com
Note: "sysconfig -a" command will give you detailed information of filer configuration including disk, shelf details

4. How to check for hardware configuration errors on Netapp: 

The command "sysconfig -c" does a configuration check and reports errors if any.
filer> sysconfig -c
sysconfig: There are no configuration errors.
filer>
5. How to get overall status of the Netapp Chassis (head) : 

The command "environment status chassis all" displays the power supply, fans , temperature sensor, nvram status etc.

filer> environment status chassis all
Temperature ok
PSU 1 ok
PSU 2 ok
Voltage ok
SYS FAN ok
NVRAM6-temperature-3 ok
NVRAM6-battery-3 ok.
filer>
6. How to view Netapp's software licenses :
Using the license command, you can view the license details on a netapp filer. "license add " will add a license.
filer> license
                 a_sis XXXXXXX
                  cifs XXXXXXX
               cluster XXXXXXX
                   fcp not licensed
            flex_clone XXXXXXX
            flex_scale not licensed
         flexcache_nfs XXXXXXX
       gateway_hitachi not licensed
                  http XXXXXXX
                 iscsi not licensed
            multistore not licensed
      nearstore_option XXXXXXX
                   nfs XXXXXXX
            snapmirror XXXXXXX
filer>
7. How to view Netapp's current autosupport configuration :
Using the "options autosupport" command, you can view the autosupport configuration details on a netapp filer.

       
filer> options autosupport

autosupport.cifs.verbose     off
autosupport.content          complete
autosupport.doit             DONT
autosupport.enable           on
autosupport.from             bali@bali.com
autosupport.local.nht_data.enable off
autosupport.local.performance_data.enable off
autosupport.mailhost         xx.xx.xx.xxautosupport.minimal.subject.id hostname
autosupport.nht_data.enable  on
autosupport.noteto
autosupport.partner.to
autosupport.performance_data.enable on
autosupport.retry.count      15
autosupport.retry.interval   4m
autosupport.support.enable   on
autosupport.support.proxy
autosupport.support.to       autosupport@netapp.com
autosupport.support.transport smtp
autosupport.support.url      support.netapp.com/asupprod/post/1.0/postAsup
autosupport.throttle         on
autosupport.to               bali@bali.com

filer>


If you wish to change the autosupport configuration, fire like this e.g. "options autosupport.mailhost xx.xx.xx.xx"

8. How to view Netapp's RLM/BMC/SP current configuration :
Using the "bmc status" "rlm status" & "sp status" command, you can view the current managment console configuration details on a netapp filer.

filer> bmc status
        Baseboard Management Controller:
                Firmware Version:   1.1
                IPMI version:       2.0
                DHCP:               off
                BMC MAC address:    xx:xx:xx:xx:xx:xx
                IP address:         xx.xx.xx.xx
                IP mask:            xx.xx.xx.xx
                Gateway IP address: xx.xx.xx.xx
                BMC ARP interval:   10 seconds
                BMC has (1) user:   naroot
                ASUP enabled:       on
                ASUP mailhost:      xx.xx.xx.xx
                ASUP from:          bali@bali.com
                ASUP recipients:    bali@bali.com, autosupport@netapp.com
                Uptime:             275 Days, 23:44:25
      filer>
If you would like to change the configuration you can use "setup" command to update the current configuration e.g. "bmc setup"

9. How to view Netapp's aggr status:
filer> aggr status
           Aggr State           Status            Options
      aggr01_sa online          raid_dp, aggr     root
            filer>
10. How to view Netapp's vol status:
filer> vol status
Volume State           Status            Options
vol0 online          raid_dp, flex     root
vol_anv_tst online          raid_dp, flex
test_drive online          raid_dp, flex     guarantee=none
vf online          raid_dp, flex
            filer>

To be Continued....

Saturday, June 23, 2012

NetApp Daily Health-Check Commands


As a system admin its necessary to check your storage devices manually even though it is integrated with monitoring system because you will get some detailed information about the current environment status. 


Commands:

 filer> rdfile /etc/messages    
Note: If you want to see old syslog messages, use following commands /etc/messages.0; /etc/messages.1 etc.,
  filer> snapmirror status
  filer> snapvault status
  filer> vol status -f (or) aggr status -f
  filer> environment chassis list-sensors
  filer> vol status
  filer> aggr status  
  filer> df -Ah
  filer> df -h
  filer> lun show offline
 filer> sysconfig -a


Hello

Hello