Github

Overview

The procedures in this topic describe how to enable the collection of Github logging in ActiveEye.

Prerequisites

Before you can set up the Github service connector in ActiveEye, you must set up audit log streaming to S3.

For more information and to set up Github audit log streaming, refer to these AWS procedures:

• Setting up Github audit log streaming

To set up the service connectors in ActiveEye, you will need the AWS account ID that owns the bucket you are streaming the Github logs to. To find the ID, reference the following AWS procedure: Finding your AWS account ID.

note

Additionally, according to the AWS Well Architected Framework, it is best practice to use a centralized AWS logging account that is separate from the rest of the infrastructure as the owner of the logging S3 bucket. This allows permissions to be controlled, and promotes security. For ActiveEye, this is recommended but not required.

Creating a Service Connector for Github

Once you have set up log streaming, a corresponding service connector must be created in ActiveEye.

1. In ActiveEye, in the left pane, click Admin, and then click Service Connectors.
   The Service Connectors page appears.
   
2. In the upper-left corner of the page, click Add Connector.
   A list of service connectors appears.
   
3. Scroll down to the BUSINESS APPLICATIONS section, and then, in the Github subsection, click the Add Connection button.
   The Add Connector Account page appears.
4. In the Display Name box, enter a unique name.
5. In the Account Number box, enter the 12-digit AWS account ID. See the Prerequisites section of this topic for information on how to find this ID.
6. Optionally, modify the priority level in the Priority box. Raising or lowering the priority will increase or decrease the visibility of alerts related to this service connector.
7. If you do not want data ingestion to begin immediately once cloud accounts have been configured, clear the Enable Account check box. Otherwise, leave the check box selected.
8. Click Add.
   The Github service connector is created.

Creating Connector Credentials

Next, you must set up an SNS notification to trigger when Github deposits a log in the S3 bucket. This is accomplished by setting up a connector credential in ActiveEye.

note

The credential does not need to be directly associated with the source AWS accounts. It is used for the assume role operation that is used to connect to and read from the bucket.

1. In ActiveEye, in the left pane, click Admin, and then click Service Connectors.
   The Service Connectors page appears.
   
2. In the upper-right corner of the page, click Manage connectors credentials.
   The Connector Credentials page appears.
   
3. In the upper-left corner of the page, click Add Credential.
   The Add Connector Credentials page appears.
4. In the Connector Type drop-down list box, click Github.
   The Github Bucket Configuration section appears.
5. In the Display Name box, enter a unique name.
6. In the Bucket Name box, enter the name of the S3 bucket that is the Github log destination. The name that you enter must meet the following criteria:
   • Must be between 3 and 63 characters long.
   • Can consist only of lowercase letters, numbers, periods, and hyphens.
   • Must begin and end with a letter or number.
   • Must not be formatted as an IP address (for example, `192.168.5.4`).
   • Must not start with the prefix `xn--`.
   • Must not end with the suffix `-s3alias`.


7. In the Bucket Account ID box, enter the 12-digit AWS account ID of the account that owns the S3 bucket that you referenced in the Bucket Name box.
8. In the Bucket Region drop-down list box, select the AWS region in which the S3 bucket resides. For a comparison of region names and codes, reference this AWS topic: Available Regions.
9. Click Add.
   The Github connector credentials are added, and the page is refreshed.
10. Record the read-only values in the SNS Notification ARN, Pod Account ID, and External ID boxes. You will need these values in subsequent procedures.

Creating CloudFormation Stack

After configuring Github to push logs to your own S3 bucket, the following steps will need to be performed by an AWS IAM user with permissions to create CloudFormation Stacks and IAM Roles.

In this procedure, you will create a customer-managed policy named “activeeye-read-github-policy” and a role that uses that policy named “activeeye-read-github-role”. This role will have GetObject permissions on the S3 bucket of your choosing only.

1. In the AWS console, in the upper-left corner of the screen, click Services.
2. In the menu that appears, click Management & Governance, and then click CloudFormation.
   The CloudFormation page appears.
3. Click the Create stack button, and then click With new resources.
   The Create stack page appears.
   
4. With the Amazon S3 URL check box selected, depending on your environment, enter the appropriate text in the Amazon S3 URL box:
   • For a commercial environment, enter the following: `https://activeeye-cloudformation-templates.s3.amazonaws.com/activeeye-read-github.yaml`
   • For a government environment, enter the following: `https://activeeye-cloudformation-templates.s3-us-gov-west-1.amazonaws.com/activeeye-read-github.yaml`


5. Click Next.
   The Specify stack details page appears.
   
6. In the Stack name box, enter the following: `activeeye-github-read`
7. In the ActiveeyeAccountID box, enter the Pod Account ID that you recorded in the previous procedure.
8. In the GithubBucket box, enter the name of the S3 bucket receiving the log files (do not include a path, prefixes, or trailing forward slashes).
9. In the ExternalID box, enter the External ID that you recorded in the previous procedure.
note

The External ID is not a secret in any way. It is simply an additional security measure that AWS recommends when allowing third parties to interact with your AWS resources. For more information on External IDs, reference this AWS blog post: How to Use External ID When Granting Access To your AWS Resources.

10. Click Next.
    The Configure stack options page appears.
11. Do not modify the default options. Click Next.
    The Review page appears.
12. Select the I acknowledge that AWS CloudFormation might create IAM resources with custom names check box, and then click Submit.
    The stack is created, and a page displaying information about it appears.

After you complete this procedure, when ActiveEye receives a notification to pull a file from a bucket, it will be able to associate the bucket with your customer account, and will have the permissions to read the file from the S3 bucket.

Enable Notifications To ActiveEye

Next, you will set up an SNS notification to trigger when a logfile is ready for ActiveEye to consume.

Create an S3 Bucket Event Notification

1. In the AWS console, in the upper-left corner of the screen, click Services.
2. In the menu that appears, click Storage, and then click S3.
   The Amazon S3 page appears, displaying a table of buckets.
3. Select the name of the bucket for which you will enable event notification.
   A page displaying details about the bucket appears.
4. Select the Properties tab, and then, in the Event notifications section, select Create event notification.
   The Create event notification page appears.
   
5. In the General Configuration section, in the Event Name box, enter `ActiveEye Github Notification`
6. In the Event Types section, select the All object create events check box.
   
7. In the Destination section, select the SNS topic check box, then select the Enter SNS topic ARN check box, and then, in the SNS topic box, enter the SNS Notification ARN value that you recorded in the Creating Connector Credentials for each Service Connector procedure.
   
8. Click Save changes.
   Notifications are enabled for the bucket.

Verifying ActiveEye Event Ingestion

Data should now begin flowing into ActiveEye within a few minutes, depending on how frequently Github writes logfiles.

To verify that data is flowing into ActiveEye from Github, in ActiveEye, select INVESTIGATE in the left pane, and then, in the Pre-Defined Queries section, in the Cloud Infrastructure subsection, select Overview. Scroll down to the bottom of the Dashboard page that appears and, in the table, look for events with the prefix Github. If these events appear, data is flowing. Note that this page does not automatically refresh.

If you suspect an issue with the data flow, follow these troubleshooting steps.

Troubleshooting

If events have not started flowing into ActiveEye, verify the following:

1. Recent Github logfiles are appearing in the S3 bucket.
2. The CloudFormation stack was successfully created with the correct bucket name in the AWS account that “owns” the S3 bucket.
3. The role activeeye-read-github-role and policy activeeye-read-github-policy appear in IAM in the AWS account that owns the S3 bucket.
4. An SNS event notification is configured on the S3 bucket with the proper ARN without a bucket path prefix or the correct bucket path prefix.
5. In the ActiveEye connector credential record, verify that the bucket name is spelled correctly, and that the AWS account ID of the S3 bucket owner and the External ID match those used in the CloudFormation stack.
6. In the ActiveEye service connector record, verify that the AWS account ID matches the AWS account that owns the Github logging bucket.
7. When the steps above have been completed, wait until at least 10 minutes have elapsed, and then check for results.

Contact ActiveEye Engineering for further help troubleshooting this issue. For fastest resolution, please provide screenshots of the AWS configurations mentioned above.