Eunsung, Could you generate a list of NERSC' GridFTP hostnames/IPs that you expect to work with? Thanks, Ben On Jul 14, 2014, at 2:51 PM, Jung, Eun Sung <[email protected]> wrote:

Hi Ben,

As of now, we will plan to do only GridFTP experiments with DTNs at NERSC.

Regards, Eunsung

On 7/14/14, 11:53 AM, "Allen, Benjamin S." <[email protected]> wrote:

...

Raj, et al.:

What firewall rules need to be put in place for external access to this testbed (both inbound and outbound)?

Do you need just the GridFTP set of ports? Do you have specific external DTNs you’re testing with, so we don’t have to open this testbed’s GridFTP services up to the entire public internet?

Thanks,

Ben

On Jul 8, 2014, at 10:59 AM, Jung, Eun Sung <[email protected]> wrote:

...

Ok, thank you.

-Eunsung

On 7/8/14, 10:54 AM, "Allen, Benjamin S." <[email protected]> wrote:

...

Hi Eun Sung,

...

* Questions regarding the testbed for baseline testing. - Is there any additional node which can play a role of DTN except two 8 file-server nodes?

In the proposed testbed no. Just to be clear I’m proposing 16 file servers, split into two groups. The 16 file servers are backed by two DDN S2A9900 racks of storage. The groups of 8 are to be defined by the file servers connected to the same backing DDN S2A9900

The idea for the baseline testing would be to use one or more of the 8 file servers from one group as clients across the network to the other group of 8 file servers. Thus simulating DTNs accessing a shared filesystem over a network.

...

- If so, all nodes including fs nodes and a DTN node share the same network?

All 16 file servers will be on the same 10Gb Myrinet network. Each file server will also have a 1GbE public interface, and a 1GbE internal management interface. The same configuration as the Petrel testbed.

The public interfaces will be heavily firewall’ed by default. We’ll explicitly allow inbound and outbound traffic as requested. Ideally we can get the list of firewall rules from Raj, Mike and you in by tomorrow. If you only need standard GridFTP ports opened, please let us know that.

...

* Questions regarding the testbed for storage-aware large data transfers - Do you mean a single file server by a single file system?

I mean a single shared filesystem (GPFS) across the 16 file servers. For testing using GPFS’ data locality hints.

...

- Once a file system created, can we adjust the number of file servers later? For example, up to 32 servers, 2 servers per fs node.

You will have 16 physical file servers. You’ll be able to reconfigure them how you see fit within reason. This includes destroying and recreating the GPFS if required.

By default all 51 LUNs (8+2p RAID6 LUNs) per DDN S2A9900 rack are presented to each of the 8 attached file servers via directly connected DDR Infiniband. We can restrict or group which LUNs are presented to which of the 8 file servers to force further data locality if you need to simulate a less ideal storage configuration.

Please let me know if you have further questions,

Ben

...

On 7/7/14, 11:44 PM, "Raj Kettimuthu" <[email protected]> wrote:

...

Thanks Ben. This sounds good to me.

Eunsung, Do you have any concerns/questions?

Raj

Sent from my iPhone

...

On Jul 7, 2014, at 4:05 PM, "Allen, Benjamin S." <[email protected]> wrote:

Hi Raj,

To summarize the requirements for everyone CC¹ed your two objectives are:

1. Baseline testing a traditional DTN which accesses the shared filesystem over a network fabric, compared to a DTN directly on a storage controller. The JLSE file servers provide reasonably facsimiles for embedded storage controllers as they have a point-to-point DDR IB connection to the D2A9900 controllers. They¹re also lower powered servers so should simulate the lower resources available on an embedded storage controller virtual machine.

2. Storage-aware Large Data Transfers: Using data locality information from GPFS to direct GridFTP to the DTN which has local block access to the data. Thus allowing GridFTP to be more efficient in its use of the storage network (Myrinet in this case). A good analogy is Hadoop¹s HDFS, but for data transfer instead of compute.

Please correct or expand the above as needed.

Due to the Petrel testbed being setup as a single file system, and being a pseudo production testbed for Globus Share work (i.e. we can't just destroy the existing globus-fs0 and rebuild it), I think the best course of action to support the above work is to setup two 8 file-server filesystems on a new Myrinet fabric. We¹ll initially create a file system on one (or optionally both) sets of 8 file servers. This will let you use opposing file servers for the baseline testing across a network fabric. This testbed would be similar in specs (same hardware, same networking) as the Petrel testbed.

Once that testing is complete, we will recreate the GPFS as a single file system so you can experiment with data locality. Without further configuration this will give you two silos of data locality (two S2A9900 racks). If you want to further worsen the test cases, we can limit which LUNs are presented to which file server, lowering the number of file-servers with block access to a given set of data.

It looks like we could have this testbed put together by early next week, as we have a near ready set of 16 file servers available. I¹m also suggesting we use this new testbed as a more general resource for those that need GPFS but can withstand the file system being destroyed. I believe this fits Bill¹s project needs, and he has similar goals in testing data locality with GPFS.

All, please let me know if you have any comments or concerns.

Thanks,

Ben