Best tools for S3 : Amazon Web Service(AWS)

s3

All those who are using AWS might have a requirement to download\sync\upload etc with your S3 bucket.

There are several ways to achieve that which includes some third party tool and even there are some ways via Visual Studio.

Let us check them one by one, we will take the example of downloading entire S3 bucket using below tools:

AWS CLI

The AWS Command Line Interface is a unified tool that provides a consistent interface for interacting with all parts of AWS.

Install AWS CLI using command sudo pip install awscli and then follow below command to download entire S3 bucket:

aws s3 sync s3://mybucket 

Using s3cmd and S3Express

s3cmd is a third party tool which is a Command Line S3 Client and Backup for Linux and Mac and S3Express is Command Line S3 Client and S3 Backup for Windows.

You can download it from above link.

Once downloaded you can configure it as below:

s3cmd --configure
s3cmd sync s3://bucketnamehere/folder /destination/folder

You can even create and remove buckets using above tools. Have a look here for more details.

Using Visual Studio

You can download entire S3 bucket using Visual Studio. For that, you would be required to download AWS toolkit for visual studio which can be downloaded from here.

Once it is installed, go to Visual Studio – AWS Explorer – S3 – Your bucket – Double click

In the window, you will be able to select all files. Right click and download files.

Using Cyberduck

One more awesome tools for this is Cyberduck.

Cyberduck is a Libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift browser for Mac and Windows.

It just requires your AWS credentials and you can use the simple interface to download/upload/sync any of your buckets/folders/files.

Using Rclone

Rclone is a command line program to sync files and directories to and from

  • Google Drive
  • Amazon S3
  • Openstack Swift / Rackspace cloud files / Memset Memstore
  • Dropbox
  • Google Cloud Storage
  • Amazon Drive
  • Microsoft OneDrive
  • Hubic
  • Backblaze B2
  • Yandex Disk
  • SFTP
  • The local filesystem

You can sync your S3 bucket by giving below commands:

rclone sync /home/local/directory remote:bucket

Using S3Browser

S3 Browser is also one tool which can be used for above purpose. It is windows only tool.

S3 Browser is a freeware Windows client for Amazon S3 and Amazon CloudFront. Amazon S3 provides a simple web-services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. Amazon CloudFront is a content delivery network (CDN). It can be used to deliver your files using a global network of edge locations.

If you know any other awesome tools then do share it here.

Happy learning.

Advertisements

Amazon Rekognition : The rise of Artificial intelligence

reko14

By looking at the word Rekognition you would feel as if it is a wrongly spelled word 🙂 It is nothing but an awesome gift from Amazon to us.

First let us see some background, I attended a training of Artificial intelligence some days back and in that training, we were shown below picture and were told to recognize what it is:

reko9

Within a second everyone said it is an elephant. We can recognize this elephant within seconds because we have years of experience of learning.

Then trainer asked what if a system wants to recognize it within seconds, is it possible? It might be very difficult because if we compare our years of experience with a system’s experience then there is a huge difference, we will have to teach the system to identify what is there in an image.

But the answer of trainer’s question is yes, it is possible with the help of AI. There are millions of images already there on the internet which can be used for Big Data or for AI.

Let us get back to Amazon Rekognition.

It was introduced in re:Invent of Amazon in 2016:

reko1

So basically it takes the image as input and returns data which can be used for different analysis purpose.

So what is Amazon Rekognition?

Amazon Rekognition is a service that makes it easy to add image analysis to your applications.

With Amazon Rekognition you can:

  • Detect objects
  • Detect scenes
  • Detect faces
  • Search and compare faces
  • Identify inappropriate content in images

Amazon Rekognition provides:

reko3

  • Object and Scene Detection
  • Image Moderation
  • Facial Analysis
  • Face Comparison
  • Facial Recognition

Let us see some use cases for different features of Amazon Rekognition:

Facial Recognition:

reko8

Sentiment Analysis:

reko7

Face-Based User Verification:

reko6

Image Moderation:

reko5

Searchable Image Library:

reko4

Interesting isn’t it?

Below are few screenshots of the actual Amazon Rekognition application:

reko10

reko11

reko12

You can see more details about the API and step by step process here if you want to know more.

It may remind you about the God’s eye from the movie The Fast & furious 7\8. That day is near as well with the rise of AI along with AWS 🙂

Happy learning.