topic Re: Rehost - Minimize Size of Vaults in Windchill

Rehost - Minimize Size of Vaults

BenPerry — Wed, 02 May 2018 12:04:07 GMT

I am looking for a way to minimize the footprint of our rehosted test & development sites. Currently the 50 GB DB is exported from Production and imported into test/dev sites. And then on our Unix server, we use the rsync command to move the application files and vault files

Re: Rehost - Minimize Size of Vaults

BineshKumar1 — Mon, 30 Nov 2015 13:39:36 GMT

There are several approaches which you could use based on the technology you are using for vaults

Approach of copying only the delta data which has been added after the last refresh is what we commonly follow. You can write a SQL to find out which all files were modified after your last refresh and provide this as an input to your rsync using --files-from=FILE flag.
If you are using VMs and Use VNX snapshot of production and attach it to your test environments - https://community.emc.com/docs/DOC-24251
Going by the logic that Windchill would write only to folders which has read-only flag disabled, you can share your read-only folders between environments. You can mark/verify whether the folders are read-only through SQLs. Indeed, this is risky and you should really put all sort of checks in place to prevent a data corruption

Thank you

Binesh Kumar

Re: Rehost - Minimize Size of Vaults

BenPerry — Mon, 30 Nov 2015 14:56:15 GMT

Binesh,

I have a few follow-up questions. I'm matching your bullets points.

Delta sync. We rehost every month. But we want perhaps 1 year of data. But still this doesn't get at the root issue. How do I get rid of all of the data from > 1 year? Just don't copy it over? But it is still referenced in the DB. So the objects exist, but the primary content does not exist in the vault, giving errors.
Snapshot. I'm not sure how this will help.
Share read-only vault folders. I agree it seems risky, and would not prefer this.

Re: Rehost - Minimize Size of Vaults

RandyJones — Mon, 30 Nov 2015 16:07:23 GMT

You don't specify what "Unix" you are running. If you are running Solaris you can take advantage of zfs snapshots/clones and zones. We are running Solaris and have the following:

1 server with zones
- production zone
- test/dev 1 zone
- test/dev 2 zone
- test/dev 3 zone
zfs filesystems for
- Windchill install
- Windchill vaults
  - master vault
    - only on the master server
  - cache vault
  - replica vault
    - on the replica servers
- Windchill oracle database
- oracle install

We snapshot and then clone the production Windchill filesystems and use these for the test/dev zones. This takes a minimal amount of disk space. Disk space is only used when the cloned filesystem diverges from the original clone. In my typical usage case the oracle tables are by far the largest "divergers".

If you didn't want to run test/dev on same server as production you still could snapshot/clone the production filesystems and then nfs mount them on the test/dev server(s).

Re: Rehost - Minimize Size of Vaults

BenPerry — Mon, 30 Nov 2015 16:13:50 GMT

We're using Red Hat Linux. Since I'm not really a server admin, I'm confused how your divergent vault is so small. Your test/dev vaults point to Production vault?

Right now what we do is have a separate 1.5 TB for the test/dev site vault, which is a copy of Production vault. That way it is completely separated. If we were able to read-only from the Production vault...but put new data into the test/dev vault, that would be great. I'm not sure how to do that though.

Re: Rehost - Minimize Size of Vaults

RandyJones — Mon, 30 Nov 2015 16:59:17 GMT

Ben Perry wrote:

We're using Red Hat Linux. Since I'm not really a server admin, I'm confused how your divergent vault is so small. Your test/dev vaults point to Production vault?

My test/dev points to cloned snapshots of the Production filesystems: vaults, Windchill install, oracle tables, oracle install

http://docs.oracle.com/cd/E19253-01/819-5461/6n7ht6r4n/index.html

ZFS - Wikipedia, the free encyclopedia

Right now what we do is have a separate 1.5 TB for the test/dev site vault, which is a copy of Production vault. That way it is completely separated. If we were able to read-only from the Production vault...but put new data into the test/dev vault, that would be great. I'm not sure how to do that though.

A cloned zfs filesystem is a writable filesystem. When you write to these file systems you are not writing to the original (Production).

Re: Rehost - Minimize Size of Vaults

TomU — Mon, 30 Nov 2015 18:19:54 GMT

Here is an interesting article on ZFS. Fun stuff.

http://www.smbitjournal.com/2014/05/the-cult-of-zfs/

Re: Rehost - Minimize Size of Vaults

TomU — Tue, 01 Dec 2015 18:21:12 GMT

Ben Perry‌,

I'm wondering if the "Selective Content Settings" in the Windchill Rehost Tool (3.0) would do what you're looking for. From the rehost.properties file:

########################### Selective Content Settings #############################

# These properties identify a set of files in vault folders in a

# staging location that will be copied to the target system

# target.content.assembly.configspec could be:

# As Stored (default value), Latest,

# Latest|STATE (such as, Latest|In Work),

# Baseline|Number(such as, Baseline|0001),

# or Promotion|Number(for e.g. Promotion|0001)

# Only required dependencies and family tables are included by default

# to include all, set the corresponding property to true.

####################################################################################

# target.content.assembly.filename.0=t2_assly.asm

# target.content.assembly.version.0=B.1

# target.content.assembly.configspec.0=Latest

# target.content.assembly.includeAllDependencies.0=false

# target.content.assembly.includeAllFamilyTables.0=false

# target.content.assembly.viewables.0=true

# target.content.assembly.drawings.0=true

# target.content.assembly.includesource.0= false

# target.content.assembly.includeimage.0=false

Re: Rehost - Minimize Size of Vaults

BenPerry — Tue, 01 Dec 2015 19:17:58 GMT

Tom,

Selective Content is kind of what I was thinking in the first place. But this ZFS thing is pretty interesting. Using ZFS it seems we could have access to the entire 1.5 TB of data without actually making a separate copy of all 1.5 TB.

I'm going to talk with our Unix admins and see if this is something we could get set up with Red Hat.

Anyone else using ZFS, or something similar to what Randy has mentioned?

Re: Rehost - Minimize Size of Vaults

BineshKumar1 — Tue, 01 Dec 2015 20:44:29 GMT

Hello Ben,

Sorry for the delayed response. The best way to get the list of files is by running a query on application data connecting to mastertables/fvitems/fvfolders and get a list of files which are <1 year and is necessary. Since you are doing it on a monthly basis, i think you can figure out a query based on the errors you get.

Regarding the snapshot method, we are using vfilers for the vaults storage, it provides snapshots/snapvaults to share the copy of the vault and to enable DR. I will message you a document on this.

Thank you

Binesh Kumar

Medtronic - MITG

Re: Rehost - Minimize Size of Vaults

JoãoGonçalves — Wed, 02 Mar 2016 18:09:11 GMT

One approach to this issue is not to 'migrate' the exiting vaults.

In your new environment, your existing vault, would link to the vault path in 'production server'.

You would put this vault as read-only.

You would create a new vault mounted in the new environment to be able to create new data.

If this new environment has only READ access (windows/OS permission) to path of 'production' folders, it is impossible to delete or broke anything.

Re: Rehost - Minimize Size of Vaults

BenPerry — Thu, 03 Mar 2016 13:39:10 GMT

Joao,

That is interesting. What are some of the key steps that you take to execute that scenario? I understand at the high level, but are there some key settings or steps you need to take?