Provides infrastructure layer compute capabilities, including both bare metal and virtual servers with various optimizations includins compute, memory, IO, and disk. Also supports accelerations options such as GPUs, FPGAs, Inferentia and Trainium.
Provides image recognition capability for images (in batch or real-time) and video that provides a analysis of the content such as real-world objects, faces, celebrities, and path mapping.
Provides a publish/subscribe notification service with multiple subscription types including Amazon Simple Queue Service (SQS), Amazon Kinesis Data Firehose, AWS Lambda, generic HTTPS endpoints, SMS and email.
A serverless, fully-managed, message queue service that supports producing, store, and consuming messages and enables loose coupling between applications.
Provides private networking capability spanning multiple availability zones and supporting subnets, routing, network access control groups, security groups and gateways.
Provides tracing of service invocations in distributed applications for observability, allowing users to diagnose issues or optimize their service interactions.
All about Cloud, mostly about Amazon Web Services (AWS)
Unable to download Amazon CloudWatch Logs?
2017-03-22 / 455 words / 3 minutes
It is hard working with large log files in Amazon CloudWatch Logs. Sometimes it’s better to be able to work with your favorite text editor. Power users may want to take advantage of command line tools like awk, sed, and grep. Unfortunately, the CloudWatch AWS Console doesn’t allow you to download log streams. Fortunately, this functionality is available from the AWS API and AWS Command Line Interface (CLI). Yet it can be tricky to download cloudwatch logs when using the Linux or Max OS X AWS Command Line Tools due to the naming convention for CloudWatch Log Stream Names. This post will show you how!
The CloudWatch Logs Command Line Interface
The AWS CLI command to use is: aws logs get-log-events
The CLI help text provides a nice description:
GET-LOG-EVENTS() GET-LOG-EVENTS()
NAME
get-log-events -
DESCRIPTION
Lists log events from the specified log stream. You can list all the
log events or filter using a time range.
By default, this operation returns as many log events as can fit in a
response size of 1MB (up to 10,000 log events). If the results include
tokens, there are more log events available. You can get additional log
events by specifying one of the tokens in a subsequent call.
See also: AWS API Documentation
SYNOPSIS
get-log-events
--log-group-name
--log-stream-name
[--start-time ]
[--end-time ]
[--next-token ]
[--limit ]
[--start-from-head | --no-start-from-head]
[--cli-input-json ]
[--generate-cli-skeleton ]
So based on this information I went to CloudWatch and noted the Log Group Name and the Log Stream Name and tried the following command:
An error occurred (ResourceNotFoundException) when calling the GetLogEvents operation: The specified log stream does not exist.
I checked the CloudWatch console. Cut and paste the names from it into the command line, but still saw the same error. Examining the Log Stream Name a little more closely, I then suspected that I had discovered the problem! I thought the text $LATEST was being interpreted by the shell on Mac OS X as a shell variable, and substituted.
Running the command again with the –debug option revealed the line:
2017-03-21 21:42:56,583 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'2017/03/21/[]3f0f582c1fa6c47408d233d23fa64873' for parameter "log_stream_name": u'2017/03/21/[] 3f0f582c1fa6c47408d233d23fa64873'
Confirmation!
How to Download CloudWatch Logs
One solution was to use single quotes (‘) around the names, which prevents the shell from doing variable substitution. You can see how the syntax highlighting has identified the text within single quotes as a string value:
All data and information provided on this site is for informational
purposes only. cloudninja.cloud makes no representations as to accuracy,
completeness, currentness, suitability, or validity of any information
on this site and will not be liable for any errors, omissions, or
delays in this information or any losses, injuries, or damages
arising from its display or use. All information is provided on an
as-is basis.
This is a personal weblog. The opinions expressed here represent my
own and not those of my employer. My opinions may change over time.
