The Cloud-First Mindset

Across every industry, cloud-native businesses are disrupting legacy institutions that have yet to transform traditional IT platforms.

A Cloud Security Strategy for the Modern World

In the borderless and elastic world of the cloud, achieving your security and compliance objectives requires a modern, agile, and autonomous security strategy that fosters a culture of security ownership across the entire organization.

Building Strength Through Partnership

The cloud partner ecosystem is changing. The days when organizations could act as a “Jack of all trades, master of none” are over. Legacy IT resellers are going the way of the dinosaur in favor of partners who can deliver clear value-add with a track record of transformative success.

11 AWS Snowball Planning Considerations

Data transfer/migration is a key consideration in any organization’s decision to move into the cloud.

If a sound strategy is applied, migration of on-premise data to the cloud is usually a seamless process. When an organization fails to do so, however, it risks running into challenges stemming from deficiencies in technical resources, inadequate planning, and/or incompatibility with legacy systems, to name a few.

Data transfer via AWS Snowball is no exception. If performed incorrectly or out of order, some of the seemingly insignificant tasks related to the data migration process can become substantial obstacles that adversely affect a timeline.  The AWS Snowball device can be simple to use if one is familiar with other AWS data transfer services and/or follows all of the steps provided in the AWS Snowball User Guide. However, neglecting a single step can greatly encumber an otherwise ordinary data transfer process.

According to AWS on its service:

“AWS Snowball is used to transport terabytes or petabytes of data to and from AWS, or who want to access the storage and compute power of the AWS Cloud locally and cost effectively in places where connecting to the internet might not be an option.”

AWS

When preparing to migrate data from on-premises storage into AWS via a Snowball device, an organization should be aware of the importance of 11 easily overlooked tasks and considerations associated with planning for the data move. They are as follows:

1. Understanding the specifics of the data being moved to the cloud.

Ensure that it is compatible and can transfer seamlessly to the cloud via AWS Snowball. Follow a cloud migration model to help layout specific details and avoid surprises during the data transfer process.

2. Verifying and validating the amount of data being transferred.

Snowball is intended for large data transfers (over 10 terabytes). Using it for smaller data transfers is not a cost-effective option.

3. Verifying that the workstation meets the minimum requirement for the data transfer.

It should have a 16-core processor, 16 MB of RAM, and a RJ45 or SPF+ network connection.

4. Performing a data transfer test on the workstation an organization plans to use to complete the task.

This will not only equip the organization with an understanding of the amount of time needed to perform the transfer but will provide an opportunity to try various methods of transferring data. Additionally, it will assist with estimating the time the Snowball device will need to be in the organization’s possession, as well as its associated cost.

NOTE: The Snowball Client must be downloaded and installed before this step is performed.

5. Creating a specific administrative IAM user account for the data transfer process via the management console.

This account will be used to order, track, create and manage Snowball Import/Export jobs and return the device to AWS.

NOTE: It is important to avoid using personal IAM user accounts if individuals will be responsible for ordering the device and performing the data transfer.

6. Following the “Object Key Naming convention” when creating S3 buckets.

It is also important to confirm that the selected S3 bucket name aligns with the expectations of the stakeholders.

7. Confirming the point of contact/s and shipping address for the Snowball device.

This is especially important if the individual ordering the device is different from the one performing the data transfer.

8. Setting up SNS notifications to help track the stages of the snowball job.

This will keep the stakeholders informed of the shipping status and the importing of data to the S3 bucket.

9. Being aware of how holidays could affect the progress or process of the data-transfer timeline.

This is important because additional costs are accrued 10 days after the Snowball is delivered.

10. Considering the organization’s administrative processes that might hinder or delay the data transfer process.

By factoring in internal processes (e.g., Change Request management, stakeholder buy-in, technical change moratoriums, etc.) into the timeframe it will take to receive the device, start the job, and ship it back to AWS can help prevent unnecessary fees.

NOTE: The Snowball device has no additional cost if it is returned within 10 days from the date it is received. Following that time, however, a daily late fee of $15 is applied until the date AWS receives it.

11. Keeping the original source data intact till the data import is confirmed.

It is very important that source data remain intact until the Snowball device has been returned to AWS, the data import has been completed, and the customer has validated the data in the S3 bucket(s).

Transferring data from on-premises to an AWS Snowball can be an uneventful endeavor when thorough planning is done in advance of ordering the device. Taking these 11 planning tasks and considerations into account is essential to eliminating some of the potential headaches and stress occasionally associated with this type of activity.

Refer to AWS Snowball Documentation for additional information and specific instructions not covered in this article.

If you or your organization has more questions, reach out to us at sales@effectual.com.

Amazon Web Service as a Data Lake

“Cloud,” “Machine Learning,” “Serverless,” “DevOps,” – technical terms utilized as buzzwords by marketing to get people excited, interested, and invested in the world of cloud architecture.

And now we have a new one – “Data Lake.” So, what is it? Why do we care? And how are lakes better than rivers and oceans? For one, it might be harder to get swept away by the current in a lake (literally, not metaphorically).

A Data Lake is a place where data is stored regardless of type – structured or unstructured. That data can then have analytics or queries ran against them. An allegory to a data lake is the internet itself. The internet, by design, is a bunch of servers labeled by IP addresses for them to communicate with each other. Search Engine web crawlers visit websites associated with these servers, accumulating data that can then be analyzed with complex algorithms. The results allow a person to type in a few words into a Search Engine and receive the most relatable information. This type of indiscriminate data accumulation and the presentation of context-relatable results is the goal of data lake utilization.

However, for anyone who wants to manage and present data in such a manner, they first need a data store to create their data lake. A prime example of such a store is Amazon S3 (Simple Storage Service) where documents, images, files, and other objects are stored indiscriminately. Have logs from servers and services from your cloud environments? Dump them here. Do you have documentation that is related to one subject, but is in different formats? Place them in S3. The file type does not really matter for a data lake.

ElasticSearch can load data from S3, indexing your data through algorithms you define and providing ways to read and access that data with your own queries. It is a service designed to provide customers with search capability without the need to build your own searching algorithms.

Athena is a “serverless interactive query service.” What does this mean? It means I can load countless CSVs into S3 buckets and have Athena return queried data as a data table output. Think database queries without the database server. Practically, you would need to implement cost management techniques (such as data partitioning) to limit the ingestion costs per query as you are charged on the amount of data read in a query.

Macie is an AWS service that ingests logs and content from all over AWS and analyzes that data for security risks. From personal identity information in S3 buckets to high-risk IAM Users, Macie is an example of what types of analysis and visualization you can do when you have a data lake.

These are just some examples of how to augment your data in the cloud. S3, by itself, is already a data lake – ‘infinite’, unorganized, and unstructured data storage. And the service already is hooked into numerous other AWS services. Data lake is here to stay and is a mere stepping stone to utilizing the full suite of technologies available now and in the future. Start with S3, add your data files, and use Lambda, ElasticSearch, Athena, and traditional web pages to display the results of those services. No servers, no OS configurations or security concerns; just development of queries, lambda functions, API calls, and data presentation – serverless.

Our team is building and managing data lakes and the associated capabilities for multiple organizations and can help yours as well. Reach out to our team at sales@effectual.com for some initial discovery.

When Best Efforts Aren’t Good Enough

“Have you tried rebooting it?” There was a time, not so long ago, when that was the first question a technician would ask when attempting to resolve an issue with a PC or a server that evolved from PCs. This was not limited to servers; IT appliances, network equipment, and other computing devices could all be expected to behave oddly if not regularly rebooted.

Adopting DevOps Methodology

DevOps methodology requires seeing engineering and IT more holistically. They work together throughout the product lifecycle, with some organizations merging them into a single team.

DevOps: Defusing Interdepartmental Tension

Cloud migration is about more than just “lifting and shifting” applications and data or modernizing applications for the cloud, it’s about transforming the culture of your entire organization.