DATA MIGRATION

Data migration is a key element to consider when adopting any new system, either through purchase or new development. One would think that any two systems that maintain the same sort of data must have performed similar tasks. Therefore, information from one system should map to the other with ease. However, this is rarely the case. Although migrating data can be a fairly time-consuming process, the benefits can be worth the cost for those that "live and die" by trends in data. Additionally, old applications need not be maintained. We includes a discussion of the following topics:  Data migration definition, Migrating legacy data decision, and How to migrate data. What is data migration? Some key terms in understanding data migration are: Legacy data is the recorded information that exists in your current storage system, and can include database records, spreadsheets, text files, scanned images and paper documents. All these data formats can be migrated to a new system. Data migration is the process of importing legacy data to a new system. This can involve entering the data manually, moving disk files from one folder (or computer) to another, database insert queries, developing custom software, or other methods. The specific method used for any particular system depends entirely on the systems involved and the nature and state of the data being migrated. Data cleansing is the process of preparing legacy data for migration to a new system. Because the architecture and storage method of new or updated systems are usually quite different, legacy data often does not meet the criteria set by the new system, and must be modified prior to migration. For example, the legacy system may have allowed data to be entered in a way that is incompatible with the new system. Architecture differences, design flaws in the legacy system, or other factors can also render the data unfit for migration in its present state. The data cleansing process manipulates, or cleans, the legacy data so it conforms to the new system's requirements. Do we migrate legacy data? The "Do we migrate legacy data?" question has been asked ever since the first companies put data in one repository and decided to change systems. Here are some commonly asked questions: Should we bring the data over to the new system? If so, should we bring all or part of the data? If just part, which parts? - based on creation date? based on status of case open or closed?÷or some combination of these? If we choose to bring over data on or after a certain date, which should it be - last 3 months, last 6 months, last year?...should it be all the data? Should we filter in or out specific data? Are our desired criteria extractable from the existing database? Are the desired fields of data importable into the new system? Is data migration from our legacy system included in the purchase of the new system? If not, do we have the expertise in-house to script an automated process to migrate the data? If not, will we hire someone to do this? Will we perform this task manually? When deciding on data migration, all factors should be examined before making the assumption that the whole dataset or none of the dataset should be moved over to the new system. The proof is in whether these data records will be used and acted upon when the new system and process is in place. Two key variables to consider in deciding on data migration include data volume and data value. Data volume is the easiest variable in the decision process. How many data records are we talking about: 1000, 10,000, 100,000, 250,000? How many are expected to come into the new system on a weekly/monthly basis to replenish this supply? Check to see if there are any technical barriers to bringing over a certain amount of data and also if large databases will affect performance of system functions like searching. If not, then 10 records or 100,000 records should not make any difference. If volume is low, then it may be well worth doing a migration so there is some database for users and for trend analysis. If volume is high, then it may make sense to examine the age/value of the data and start filtering on certain criteria. Data value is a much harder variable in the decision process. Many times there are different perceptions concerning what value the existing data provide. If users are not working with older data in the current system, chances are they may not work with older data in the new system even with improved search functionality. If migrating, you may want to look at shorter-term date parameters - why bog down a system's performance with data that are never used? Criteria, as discussed in the questions above, can be date parameters, but can also include other factors. Extracting the exact data based on some of these factors will depend on the abilities of your current system and database as well as the ability to write the detailed extraction script. Keeping it simple when possible is the best approach. However, there may be circumstances where filtering data may make sense. Once you have determined which data you want to migrate, then determining what parts of the data record will also be important. How do we migrate data? Once the decision is made to perform data migration and before migration can begin the following analyses must be performed: Analyze and define source structure (structure of data in the legacy system) Analyze and define target structure (structure of data in the new system) Perform field mapping (mapping between the source and target structure with data cleansing, if necessary) Define the migration process (automated vs. manual) To analyze and define source and target structures, analysis must be performed on the existing system as well as the new system to understand how it works, who uses it, and what they use it for. A good starting point for gathering this information is in the existing documentation for each system. This documentation could take the form of the original specifications for the application, as well as the systems design and documentation produced once the application was completed. Often this information will be missing or incomplete with legacy applications, because there may be some time between when the application was first developed and now. You may also find crucial information in other forms of documentation, including guides, manuals, tutorials, and training materials that end-users may have used. Most often this type of material will provide background information on the functionality exposed to end-users but may not provide details of how the underlying processes work. For this part of the analysis, you may actually need to review and document the existing code. This may be difficult, depending on the legacy platform. For example, if data are being migrated from an AS/400 application written in RPG, assistance from an experienced RPG programmer will be required, if those skills are not available in-house. This can be an expensive part of the analysis process, because a resource to perform this analysis for you may be necessary, but it is a vital part of the process, especially if the application has been running and maintained over a long period of time. Undocumented code or fixes that are critical to the application and that have not been documented elsewhere may exist. Another key area to examine is how the data in the system are stored (i.e., in flat files, files, or tables). What fields are included in those files/tables and what indexes are in use? Also, a detailed analysis of any server processes that are running that may be related to the data must be performed (e.g., if a nightly process runs across a file and updates it from another system). Now that the source and target structures are defined, the mapping from the legacy to the target should fall into place fairly easily. Mapping should include documentation that specifically identifies fields from the legacy system mapped to fields in the target system and any necessary conversion or cleansing. Once the analysis and mapping steps are completed, the process of importing the data into the new system must be defined. This process may be a combination of automation and manual processes or may be completely automated or may be completely manual. For example, a process may: Create data extractions from the legacy system Cleanse the data extractions according to mapping guidelines Import the data extractions into the new system using that system's import feature Verify and test samplings of the data in the new system by comparing data reports to the old system Bottom Line. Data migration is a key element to consider when adopting any new system either through purchase or new development. Data migration is not a simplistic task and if not given due consideration early in the process of developing or purchasing a new system it could become a very expensive task. Migrating PM Data from the TIMS Legacy System. In TIMS, information is linked by the patient (known as client) to PM and Surveillance data. TIMS is a client-centered (or patient-centered) application. The following is a diagram of data for TIMS PM activities associated with a patient. The following types of medical data can be entered and maintained for a patient: Diagnoses - TB information about a patient's condition, disease, date, site of disease, etc. used to classify the client's TB status Previous Medical History - Information about previous medical conditions Test Data - Various tests performed for a patient including skin tests, chest x-rays, blood tests, HIV tests, bacteriology testing, and physical examination Medications - Drugs prescribed for a patient to include start time, stop time, stop reason, dosages, etc. Contacts - Information about people with whom the client has been associated and who may have been exposed to TB through that association Hospitalizations - Information about previous or current hospital care the patient has received Referrals - Information about the person or organization that sent the patient to be examined Basic demographic information is stored in the following tables in TIMS: Client - Patient's name, address, date of birth, sex, race ethnicity, etc. Address - Multiple address locations for a client Alert - Note on the client that may be worthwhile or even necessary to inform clinic staff, such as hard to treat, drug resistant, etc. Alias - Multiple names used by a client Phone - Multiple telephone numbers used/owned by a client Disposition - Status of a client, such as open or closed and if closed, then reason for closure (Moved, Lost, Unknown, etc.) Appointments - Records of appointments established for the client TIMS provides two options for extracting data: an export facility and an ad-hoc query system built into the system module of the application. The Comma Separated Value (CSV) is a text format with commas that separate field values. This is a commonly used file format recognized by the greatest number of software products, and is therefore recommended.

Types of e-Commerce

The term "e-business" was coined by Lou Gerstner, CEO of IBM.

There are mainly five types of e-commerce models:

1. Business to Consumer (B2C) - As the name suggests, it is the model involving businesses and consumers. This is the most common e-commerce segment. In this model, online businesses sell to individual consumers. When B2C started, it had a small share in the market but after 1995 its growth was exponential. The basic concept behind this type is that the online retailers and marketers can sell their products to the online consumer by using crystal clear data which is made available via various online marketing tools. E.g. An online pharmacy giving free medical consultation and selling medicines to patients is following B2C model.

2. Business to Business (B2B) - It is the largest form of e-commerce involving business of trillions of dollars. In this form, the buyers and sellers are both business entities and do not involve an individual consumer. It is like the manufacturer supplying goods to the retailer or wholesaler. E.g. Dell sells computers and other related accessories online but it is does not manufacture all those products. So, in order to sell those products, it first purchases them from different businesses i.e. the manufacturers of those products.

Benefits:

·         Encourage your businesses online

·         Products import and export

·         Determine buyers and suppliers

·         Position trade guides

3. Consumer to Consumer (C2C) - It facilitates the online transaction of goods or services between two people. Though there is no visible intermediary involved but the parties cannot carry out the transactions without the platform which is provided by the online market maker such as eBay.

4. Peer to Peer (P2P) - Though it is an e-commerce model but it is more than that. It is a technology in itself which helps people to directly share computer files and computer resources without having to go through a central web server. To use this, both sides need to install the required software so that they can communicate on the common platform. This type of e-commerce has quite low revenue generation as from the beginning it has been inclined to the free usage due to which it sometimes got entangled in cyber laws.

5. m-Commerce - It refers to the use of mobile devices for conducting the transactions. The mobile device holders can contact each other and can conduct the business. Even the web design and development companies optimize the websites to be viewed correctly on mobile devices.

There are other types of e-commerce business models too like Business to Employee (B2E), Government to Business (G2B) and Government to Citizen (G2C) but in essence they are similar to the above mentioned types. Moreover, it is not necessary that these models are dedicatedly followed in all the online business types. It may be the case that a business is using all the models or only one of them or some of them as per its needs.

DIFFERENCE BETWEEN e COMMERCE & e BUSINESS

1.     e-Commerce or Electronic Commerce encompasses those businesses offering products or services to either consumers or other businesses over the internet.

E-commerce can be broken up into two main categories:


   Online Purchasing: This business presents the customer with those technologies that make it easier for them to find data and buy commodities. The businesses encompassed in this category serve the customer by giving them the option of order placement, purchase order submission or requisition of quotes.

   Online Shopping: This involves businesses giving information to customers so that they can make a decision and buy a certain commodity from you.

To explain these two concepts lets say we go to XYZ.com and we are reviewing ABC product. When we are doing this, we are participating in Online Shopping. We then decide whether the commodity ABC is the right choice, you then add it to the shopping cart. After you have chosen whatever products we want, we then click the purchase button. As soon as we click the purchase button, we then participate in an online Purchase. This process involves all those secured pages that ask you for details such as credit card information, shipping details and other particulars.

e-Business or Electronic Business on the other hand, are those businesses that run the traditional way but also cater to the needs of online requests. An e-business status is received when  we can handle the calls, mail orders and the online activities.

Having said this,we will find that many sites although are related to e-commerce but are not e-businesses. E-commerce website only caters to selling of a commodity. Whereas an e-business has the companies activities amalgamated with the website. This means the marketing, accounting system, manufacturing, operations and other functions of the business is in contrast with the website activities.

2.     The term e-commerce is often described as outward process as it involves processes like reaching the customers, any external partner, external suppliers etc. On the other hand, e-business covers various internal processes like content management, risk management, inventory management, production management, finance management and other internal business processes.

3.     In practice, e-business usually includes e-commerce. E-commerce seeks to add revenue streams using the Worldwide Web or the Internet to build and enhance relationships with clients and partners and to improve efficiency using the Empty Vessel strategy. Often, e-commerce involves the application of knowledge management systems.

E-business is more than just e-commerce. It involves business processes spanning the entire value chain: electronic purchasing and supply chain management, processing orders electronically, handling customer service, and cooperating with business partners. Special technical standards for e-business facilitate the exchange of data between companies. E-business software solutions allow the integration of intra and inter firm business processes. E-business can be conducted using the Web, the Internet, intranets, extranets, or some combination of these.

4.     E-commerce principally involves money exchanges in the transactions. In e-business, as it is broader, it is not limited to monetary transactions. All aspects in business are included like marketing, product design, supply management, etc.

5.     E-business is more about making great products, brainstorming and giving quality service, planning about product exposure and executing it. Well, of course, e-commerce is an integral part of the e-business process but in strict terms, it is the activity of selling and buying

SUMMARY

1.     E-business is broader in scope and e-commerce is just an aspect or a subset of it.

2. E-commerce only covers business transactions such as buying and selling of goods and services   over the internet.
3. E-commerce essentially involves monetary trade while in e-business, money transactions are not necessary.
4. E-business involves marketing, product design, consumer service evaluation, and more.

POINT TO BE NOTED

Many companies have an eCommerce site but are not yet an eBusiness. eCommerce is the online selling component of a web site. eBusiness is the integration of a company's activities including products, procedures, and services with the Internet. We turn our company from a business into an eBusiness when we integrate our sales, marketing, accounting, manufacturing, and operations with our web site activities. An eBusiness uses the Internet as fully integrated channel for all business activities.

The following is an example of a company that has not yet become an eBusiness: we visit a retailer's web site and buy a shirt. When the shirt arrives it is in the wrong size. We decide to return the shirt at the store's retail outlet instead of mailing it back to the vendor. However, when we go to the store we are told that they cannot take returns from their web site. Since the web site is not integrated with the rest of their business activities this company is not yet an eBusiness. If the company had integrated their web site with their stores by providing access to their web site from within the store, by accepting exchanges for sales made online, and by training their people to support customers from/with their web site, they would be an eBusiness. eBusinesses do not consider the web site as a separate activity from their core businesses: The web site is integral to all activities at an eBusiness.

Companies who are performing Businesses to Business activities become eBusinesses when they integrate standard activities with their web site. A salesperson considers their web site a sales tool. When talking to a customer the sales person takes the customer to their web site to give product presentations, provides the customer with virtual tours of the newest products, or shows the customer how to use a tool that the customer can use to configure their products. The Marketing Department releases products on the website first, providing online product presentations, eLearning courses, and brochures. Customer Support uses the web site to host FAQ (Frequently Asked Questions), support chat lines, and moderate newsgroups. Purchasing uses the web to obtain prices on necessary components and place orders, and Shipping uses the web to schedule deliveries and notify customers of product arrival. Within an eBusiness every department within the company treats the web as an important tool they can use to move business ahead.

Is our company a business or an eBusiness? Most companies go through the following series of phases as they evolve from a business to an eBusiness.

·        Phase 0: A company doesn't understand how they can take advantage of the Internet.

• Phase 1: An outside company is hired to create a web site. The web site is static, containing little more than an on-line brochure. Employees rarely reference the site or update it with the latest information.

• Phase 2: The Company decides to integrate eCommerce on the site. Someone within the company is charged with placing product catalog on-line. Salespeople consider the web site a threat - not an integral tool to conduct business.

·        Phase 3: Everyone's internal Power Point presentations include Web Site initiatives. None of these initiatives have been executed.

• Phase 4. The company starts implementing initiatives. The web site responsibility moves in-house and is assigned to an important member of the Senior Staff.

• Phase 5: The web site becomes important to All of Senior Staff. Initiatives are implemented; employees are focused on using the web to conduct business. The company is becoming an eBusiness.