Understanding Block Storage
Our Block Storage service (which is currently available in AMS1, EWR1, NRT1, & SJC1) is a multi-tenant storage array mounted over our network via iSCSI. Learn more about the basics of deploying and attaching a volume here.
Our Block Storage service looks and smells a lot like Amazon’s EBS. One important difference is that when you connect a block storage volume on Packet to a server on Packet, you don’t have the benefit of a hypervisor layer. As such, there is a bit more you should know about how to troubleshoot or deal with any issues that may arise.
While our Block Storage product is the perfect match for a wide variety of use cases, the first and most important step is to understand when *not* to use block storage at Packet.
As a general rule of thumb, databases are not a good fit for our Block Storage, especially those that are not resilient due to minor hiccups. While uncommon, even a small loss of connectivity to some databases can cause major issues.
Each installation of our Block Storage product is made up of three physical servers, each with redundant network and power. In order to support data redundancy, each volume you create is replicated twice on the storage cluster.
The following commands will show more information about these paths:
root@block-storage:~# iscsiadm -m session
tcp:  10.144.50.243:3260,1 iqn.2013-05.com.daterainc:tc:01:sn:e5ac2dc1d7335491 (non-flash)
tcp:  10.144.34.62:3260,1 iqn.2013-05.com.daterainc:tc:01:sn:e5ac2dc1d7335491 (non-flash)
From this output, you can see the two iSCSI target IPs. If you are able to ping them fine, that means there is good network connectivity to the volume. Next, we recommend that you check DM-multipath with the following command:
root@block-storage:~# multipath -ll
volume-9674e518 (360014057ee4d5efb13a4c33a24ba765f) dm-0 DATERA,IBLOCK
size=200G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
|- 9:0:0:0 sdc 8:32 active ready running
`- 8:0:0:0 sdd 8:48 active ready running
This provides you further information about the volume.
Using the -m queue option when attaching a volume is recommended, since during instances of very brief network blips, the server will keep IO in a memory buffer until the volume is reachable again. This greatly enhances resiliency and will prevent the volume from going into read-only mode.
Although we do our best to prevent downtime, it would be a lie to say that network attached anything will have an 100% uptime, forever. There will be cases when a volume will go into read-only mode, such as if there is a longer network interruption, or there are issues with the storage cluster itself (volumes go read-only in order to protect any information that is there against possible corruption).
The best way to bring a volume back in this scenario is to detach it with
After a few seconds, re-attach it with:
packet-block-storage-attach -v -m queue
The Verbose option will show any possible issues during the attach process. If the attach command is not successful, or if it shows any problems, please contact us.