https://github.com/awslabs/aws-athena-query-federation

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own data sources and code.

https://github.com/awslabs/aws-athena-query-federation

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (13.6%) to scientific vocabulary
Last synced: 6 months ago · JSON representation

Repository

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own data sources and code.

Basic Info
  • Host: GitHub
  • Owner: awslabs
  • License: apache-2.0
  • Language: Java
  • Default Branch: master
  • Homepage:
  • Size: 72.9 MB
Statistics
  • Stars: 589
  • Watchers: 26
  • Forks: 320
  • Open Issues: 120
  • Releases: 132
Created over 6 years ago · Last pushed 6 months ago
Metadata Files
Readme Contributing License Code of conduct

README.md

Amazon Athena Query Federation

Build Status codecov

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code. This enables you to integrate with new data sources, proprietary data formats, or build in new user defined functions. Initially these customizations will be limited to the parts of a query that occur during a TableScan operation but will eventually be expanded to include other parts of the query lifecycle using the same easy to understand interface.

Athena Federated Queries are now available where Athena is supported. To use this feature, upgrade your engine version to Athena V2 in your workgroup settings. Check documentation here for more details: https://docs.aws.amazon.com/athena/latest/ug/engine-versions.html.

tldr; Get Started: 1. Ensure you have the proper permissions/policies to deploy/use Athena Federated Queries 1. Navigate to Servless Application Repository and search for "athena-federation". Be sure to check the box to show entries that require custom IAM roles. 1. Look for entries published by the "Amazon Athena Federation" author. 1. Deploy the application 1. To use Federated Queries, upgrade your engine version to Athena V2 in your workgroup settings. Check documentation here for more details: https://docs.aws.amazon.com/athena/latest/ug/engine-versions.html. 1. Run a query "show databases in `lambda:`" where is the name of the Lambda function you deployed in the previous steps.

For more information please consult:

  1. Intro Video
  2. SDK ReadMe
  3. Quick Start Guide
  4. Available Connectors
  5. Federation Features
  6. How To Build A Connector or UDF
  7. Gathering diagnostic info for support
  8. Frequently Asked Questions
  9. Common Problems
  10. Installation Pre-requisites
  11. Known Limitations & Open Issues
  12. Predicate Pushdown How-To
  13. Our Github Wiki.
  14. Java Doc

Architecture Image

We've written integrations with more than 20 databases, storage formats, and live APIs in order to refine this interface and balance flexibility with ease of use. We hope that making this SDK and initial set of connectors Open Source will allow us to continue to improve the experience and performance of Athena Query Federation.

Serverless Big Data Using AWS Lambda

Architecture Image

Queries That Span Data Stores

Imagine a hypothetical e-commerce company who's architecture uses:

  1. Payment processing in a secure VPC with transaction records stored in HBase on EMR
  2. Redis is used to store active orders so that the processing engine can get fast access to them.
  3. DocumentDB (e.g. a mongodb compatible store) for Customer account data like email address, shipping addresses, etc..
  4. Their e-commerce site using auto-scaling on Fargate with their product catalog in Amazon Aurora.
  5. Cloudwatch Logs to house the Order Processor's log events.
  6. A write-once-read-many datawarehouse on Redshift.
  7. Shipment tracking data in DynamoDB.
  8. A fleet of Drivers performing last-mile delivery while utilizing IoT enabled tablets.
  9. Advertising conversion data from a 3rd party source.

Architecture Image

Customer service agents begin receiving calls about orders 'stuck' in a weird state. Some show as pending even though they have delivered, others show as delivered but haven't actually shipped. It would be great if we could quickly run a query across this diverse architecture to understand which orders might be affected and what they have in common.

Using Amazon Athena Query Federation and many of the connectors found in this repository, our hypothetical e-commerce company would be able to run a query that:

  1. Grabs all active orders from Redis. (see athena-redis)
  2. Joins against any orders with 'WARN' or 'ERROR' events in Cloudwatch logs by using regex matching and extraction. (see athena-cloudwatch)
  3. Joins against our EC2 inventory to get the hostname(s) and status of the Order Processor(s) that logged the 'WARN' or 'ERROR'. (see athena-cmdb)
  4. Joins against DocumentDB to obtain customer contact details for the affected orders. (see athena-docdb)
  5. Joins against DynamoDB to get shipping status and tracking details. (see athena-dynamodb)
  6. Joins against HBase to get payment status for the affected orders. (see athena-hbase)

```sql WITH logs AS (SELECT logstream, message AS orderprocessorlog, Regexpextract(message, '.orderId=(\d+) .', 1) AS orderId, Regexpextract(message, '(.):.', 1) AS loglevel FROM "lambda:cloudwatch"."/var/ecommerce-engine/order-processor".alllogstreams WHERE Regexpextract(message, '(.):.', 1) != 'WARN'), activeorders AS (SELECT * FROM redis.redisdb.rediscustomerorders), orderprocessors AS (SELECT instanceid, publicipaddress, state.NAME FROM awscmdb.ec2.ec2instances), customer AS (SELECT id, email FROM docdb.customers.customerinfo), addresses AS (SELECT id, isresidential, address.street AS street FROM docdb.customers.customeraddresses), shipments AS ( SELECT orderid, shipmentid, fromunixtime(cast(shippeddate as double)) as shipmenttime, carrier FROM lambdaddb.default.ordershipments), payments AS ( SELECT "summary:orderid", "summary:status", "summary:ccid", "details:network" FROM "hbase".hbasepayments.transactions)

SELECT key AS redisorderid, customerid, customer.email AS custemail, "summary:ccid" AS creditcard, "details:network" AS CCtype, "summary:status" AS paymentstatus, status AS redisstatus, addresses.street AS streetaddress, shipments.shipmenttime as shipmenttime, shipments.carrier as shipmentcarrier, publicipaddress AS ec2orderprocessor, NAME AS ec2state, loglevel, orderprocessorlog FROM activeorders LEFT JOIN logs ON logs.orderid = activeorders.key_ LEFT JOIN orderprocessors ON logs.logstream = orderprocessors.instanceid LEFT JOIN customer ON customer.id = customerid LEFT JOIN addresses ON addresses.id = addressid LEFT JOIN shipments ON shipments.orderid = activeorders.key_ LEFT JOIN payments ON payments."summary:orderid" = activeorders.key ```

License

This project is licensed under the Apache-2.0 License.

Owner

  • Name: Amazon Web Services - Labs
  • Login: awslabs
  • Kind: organization
  • Location: Seattle, WA

AWS Labs

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 192
  • Total pull requests: 1,310
  • Average time to close issues: 12 months
  • Average time to close pull requests: 12 days
  • Total issue authors: 119
  • Total pull request authors: 50
  • Average comments per issue: 1.51
  • Average comments per pull request: 1.19
  • Merged pull requests: 899
  • Bot issues: 9
  • Bot pull requests: 883
Past Year
  • Issues: 46
  • Pull requests: 672
  • Average time to close issues: 26 days
  • Average time to close pull requests: 5 days
  • Issue authors: 36
  • Pull request authors: 28
  • Average comments per issue: 0.43
  • Average comments per pull request: 0.93
  • Merged pull requests: 419
  • Bot issues: 4
  • Bot pull requests: 451
Top Authors
Issue Authors
  • macohen (24)
  • aimethed (17)
  • dependabot[bot] (9)
  • avirtuos (5)
  • abhishekpoddar-trianz (3)
  • burhan94 (3)
  • evbo (2)
  • bishrtabbaa (2)
  • KevinGlinski (2)
  • tlierdotfr (2)
  • BhuviTheDataGuy (2)
  • pjoe (2)
  • sbrandtb (2)
  • sergei-ivanov (2)
  • Moreno-Gentili (2)
Pull Request Authors
  • dependabot[bot] (883)
  • AbdulR3hman (64)
  • aimethed (58)
  • Trianz-Akshay (47)
  • ejeffrli (40)
  • Jithendar12 (36)
  • VenkatasivareddyTR (34)
  • chngpe (30)
  • ritiktrianz (15)
  • akuzin1 (13)
  • burhan94 (7)
  • macohen (7)
  • fal-bharadwaj (6)
  • hackett123 (6)
  • MarioRial22 (6)
Top Labels
Issue Labels
enhancement (80) bug (73) question (20) dependencies (9) java (9) good first issue (2) performance (2) API (1)
Pull Request Labels
dependencies (881) java (869) requires-manual-qa (52) github_actions (8) javascript (4) enhancement (4) glue_connections_adoption (1) question (1)

Packages

  • Total packages: 9
  • Total downloads: unknown
  • Total docker downloads: 1,291,509
  • Total dependent packages: 1
    (may contain duplicates)
  • Total dependent repositories: 21
    (may contain duplicates)
  • Total versions: 685
repo1.maven.org: com.amazonaws:aws-athena-federation-sdk

The Athena Query Federation SDK defines a set of interfaces and wire protocols that you can implement to enable Athena to delegate portions of it's query execution plan to code that you deploy/write.

  • Versions: 122
  • Dependent Packages: 1
  • Dependent Repositories: 21
  • Docker Downloads: 1,291,509
Rankings
Docker downloads count: 1.9%
Dependent repos count: 5.2%
Forks count: 12.5%
Average: 13.6%
Stargazers count: 15.8%
Dependent packages count: 32.7%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:aws-athena-query-federation

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 122
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Forks count: 9.2%
Stargazers count: 12.0%
Average: 25.5%
Dependent repos count: 32.0%
Dependent packages count: 48.9%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-cloudwatch-metrics

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 71
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-aws-cmdb

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 71
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-federation-integ-test

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 71
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-cloudwatch

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 71
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-dynamodb

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 71
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-mysql

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 43
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago
repo1.maven.org: com.amazonaws:athena-jdbc

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code.

  • Versions: 43
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 33.6%
Average: 40.8%
Dependent packages count: 48.0%
Last synced: 6 months ago

Dependencies

athena-aws-cmdb/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-ec2 ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-emr ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-cloudera-hive/pom.xml maven
  • Hive:HiveJDBC42 2.6.15.1018
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • com.github.stefanbirkner:system-rules 1.19.0 test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-cloudera-impala/pom.xml maven
  • com.github.spotbugs:spotbugs-annotations RELEASE compile
  • Impala:ImpalaJDBC42 2.6.26.1031
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-cloudwatch/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-logs ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • software.amazon.awscdk:logs ${aws-cdk.version}
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
athena-cloudwatch-metrics/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-cloudwatch ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-datalakegen2/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.microsoft.sqlserver:mssql-jdbc 9.2.1.jre11
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-docdb/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.mongodb:mongo-java-driver 3.10.2
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:aws-java-sdk-docdb ${aws-sdk.version} test
  • software.amazon.awscdk:docdb ${aws-cdk.version} test
athena-dynamodb/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-dynamodb ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:DynamoDBLocal 1.11.477 test
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • org.hamcrest:hamcrest 2.1 test
  • software.amazon.awscdk:dynamodb ${aws-cdk.version} test
athena-elasticsearch/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-elasticsearch ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.httpcomponents:httpclient 4.5.13
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.elasticsearch.client:elasticsearch-rest-client 7.0.0
  • org.elasticsearch.client:elasticsearch-rest-high-level-client 7.0.0
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • junit:junit 4.13.1 test
  • org.mockito:mockito-core 3.3.3 test
  • software.amazon.awscdk:elasticsearch ${aws-cdk.version} test
athena-example/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-federation-integ-test/pom.xml maven
  • com.amazonaws:aws-java-sdk-athena ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-cloudformation ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-secretsmanager ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • com.fasterxml.jackson.core:jackson-annotations ${fasterxml.jackson.version}
  • com.fasterxml.jackson.core:jackson-core ${fasterxml.jackson.version}
  • com.google.guava:guava 29.0-jre
  • commons-cli:commons-cli 1.4
  • org.apache.commons:commons-lang3 3.9
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • org.testng:testng ${testng.version}
  • software.amazon.awscdk:athena ${aws-cdk.version}
  • software.amazon.awscdk:core ${aws-cdk.version}
  • software.amazon.awscdk:iam ${aws-cdk.version}
  • software.amazon.awscdk:lambda ${aws-cdk.version}
athena-federation-sdk/pom.xml maven
  • com.amazonaws:aws-java-sdk-athena ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-glue ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-kms ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-lambda ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-s3 ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-secretsmanager ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-core 1.2.0
  • com.google.guava:guava 29.0-jre
  • org.apache.arrow:arrow-memory-core 3.0.0
  • org.apache.arrow:arrow-memory-netty 3.0.0
  • org.apache.arrow:arrow-vector 3.0.0
  • org.apache.commons:commons-lang3 3.9
  • org.bouncycastle:bcprov-jdk15on 1.67
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version} test
  • junit:junit 4.13.1 test
  • org.apache.logging.log4j:log4j-api ${log4j2Version} test
  • org.apache.logging.log4j:log4j-core ${log4j2Version} test
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version} test
  • org.mockito:mockito-all 1.10.19 test
athena-federation-sdk-tools/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • commons-cli:commons-cli 1.4
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-google-bigquery/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.google.cloud:google-cloud-bigquery 1.87.0
  • com.google.cloud:google-cloud-resourcemanager 0.108.0-alpha
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • nl.jqno.equalsverifier:equalsverifier 3.5.5 test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-hbase/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-emr ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.hbase:hbase-client 1.4.10
  • org.apache.httpcomponents:httpclient 4.5.13
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • software.amazon.awscdk:emr ${aws-cdk.version} test
athena-hortonworks-hive/pom.xml maven
  • Hive:HiveJDBC42 2.6.15.1018
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • com.github.stefanbirkner:system-rules 1.19.0 test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-jdbc/pom.xml maven
  • com.github.spotbugs:spotbugs-annotations 4.5.3 compile
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.antlr:stringtemplate 4.0.2
  • org.apache.arrow:arrow-jdbc 2.0.0
  • org.apache.commons:commons-text 1.8
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • com.amazonaws:aws-java-sdk-redshift ${aws-sdk.version} test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
  • software.amazon.awscdk:redshift ${aws-cdk.version} test
athena-mysql/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • mysql:mysql-connector-java 8.0.17
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-neptune/pom.xml maven
  • com.amazonaws:amazon-neptune-gremlin-java-sigv4 2.1.1
  • com.amazonaws:amazon-neptune-sigv4-signer 2.1.1
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • io.netty:netty-all 4.1.42.Final
  • io.netty:netty-buffer 4.1.42.Final
  • io.netty:netty-common 4.1.42.Final
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.apache.tinkerpop:gremlin-core 3.4.8
  • org.apache.tinkerpop:gremlin-driver 3.4.8
  • org.apache.tinkerpop:gremlin-groovy 3.4.8
  • org.apache.tinkerpop:tinkergraph-gremlin 3.4.8
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-oracle/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.oracle.database.jdbc:ojdbc8 21.3.0.0
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-postgresql/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • org.postgresql:postgresql 42.3.3
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-redis/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • io.lettuce:lettuce-core 6.1.4.RELEASE
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:aws-java-sdk-elasticache ${aws-sdk.version} test
  • software.amazon.awscdk:elasticache ${aws-cdk.version} test
  • software.amazon.awscdk:glue ${aws-cdk.version} test
athena-redshift/pom.xml maven
  • com.amazon.redshift:redshift-jdbc42-no-awssdk 1.2.34.1058
  • com.amazonaws:athena-postgresql 2022.24.1
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • com.amazonaws:aws-java-sdk-redshift ${aws-sdk.version} test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
  • software.amazon.awscdk:redshift ${aws-cdk.version} test
athena-saphana/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.sap.cloud.db.jdbc:ngdbc 2.12.5
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-snowflake/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • net.snowflake:snowflake-jdbc 3.13.14
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • com.github.stefanbirkner:system-rules 1.2.0 test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
  • uk.org.webcompere:system-stubs-junit4 1.1.0 test
athena-sqlserver/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.microsoft.sqlserver:mssql-jdbc 9.2.1.jre11
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-synapse/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.microsoft.sqlserver:mssql-jdbc 9.2.1.jre11
  • org.antlr:ST4 4.3.1
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-teradata/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.24.1
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • com.amazonaws:athena-jdbc 2022.24.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.powermock:powermock-api-mockito2 2.0.9 test
  • org.powermock:powermock-core 2.0.9 test
  • org.powermock:powermock-module-junit4 2.0.9 test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
  • uk.org.webcompere:system-stubs-junit4 1.1.0 test
athena-timestream/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-java-sdk-timestreamquery 1.11.873
  • com.amazonaws:aws-java-sdk-timestreamwrite 1.11.873
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.antlr:stringtemplate 4.0.2
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
  • com.amazonaws:athena-federation-integ-test 2022.24.1 test
  • software.amazon.awscdk:timestream ${aws-cdk.version} test
athena-tpcds/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • com.teradata.tpcds:tpcds 1.2
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-udfs/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
athena-vertica/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.24.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • com.vertica.jdbc:vertica-jdbc 10.0.1-0
  • org.antlr:ST4 4.3.1
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j 1.7.30
  • org.slf4j:slf4j-api 1.7.30
pom.xml maven
  • com.amazonaws:aws-lambda-java-core 1.2.0
  • junit:junit 4.13.1 test
  • org.mockito:mockito-all 1.10.19 test
  • org.testng:testng 7.0.0 test
athena-neptune/docs/aws-glue-sample-scripts/automation/package-lock.json npm
  • aws-sdk 2.992.0
  • base64-js 1.5.1
  • buffer 4.9.2
  • events 1.1.1
  • fs 0.0.1-security
  • ieee754 1.1.13
  • isarray 1.0.0
  • jmespath 0.15.0
  • punycode 1.3.2
  • querystring 0.2.0
  • sax 1.2.1
  • url 0.10.3
  • uuid 3.3.2
  • xml2js 0.4.19
  • xmlbuilder 9.0.7
athena-neptune/docs/aws-glue-sample-scripts/automation/package.json npm
  • aws-sdk ^2.992.0
  • fs ^0.0.1-security
  • uuid ^3.3.2
.github/workflows/codeql-analysis.yml actions
  • actions/checkout v3 composite
  • github/codeql-action/analyze v2 composite
  • github/codeql-action/autobuild v2 composite
  • github/codeql-action/init v2 composite
.github/workflows/javadoc_sync.yaml actions
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
  • ad-m/github-push-action master composite
.github/workflows/maven_push.yml actions
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
.github/workflows/publish_to_maven_central.yml actions
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
athena-db2/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.47.1
  • com.ibm.db2:jcc 11.5.8.0
  • com.amazonaws:athena-federation-integ-test 2022.47.1 test
  • com.amazonaws:athena-jdbc 2022.47.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.mockito:mockito-core ${mockito.legacy.version} test
  • org.powermock:powermock-api-easymock ${powermock.version} test
  • org.powermock:powermock-api-mockito2 ${powermock.version} test
  • org.powermock:powermock-core ${powermock.version} test
  • org.powermock:powermock-module-junit4 ${powermock.version} test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-msk/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.47.1
  • com.amazonaws:aws-java-sdk-glue ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-sts ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • com.fasterxml.jackson.core:jackson-annotations ${fasterxml.jackson.version}
  • com.google.guava:guava ${guava.version}
  • org.apache.kafka:kafka-clients 3.3.1
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j2-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j ${slf4j-log4j.version}
  • org.slf4j:slf4j-api ${slf4j-log4j.version}
  • software.amazon.msk:aws-msk-iam-auth 1.1.5
  • com.amazonaws:athena-federation-integ-test 2022.47.1 test
  • com.github.stefanbirkner:system-rules ${test.system.rules.version} test
  • org.junit.support:testng-engine 1.0.4 test
  • org.mockito:mockito-core ${mockito.legacy.version} test
  • org.powermock:powermock-api-mockito2 ${powermock.version} test
  • org.powermock:powermock-core ${powermock.version} test
  • org.powermock:powermock-module-junit4 ${powermock.version} test
  • org.slf4j:slf4j-simple ${slf4j-log4j.version} test
  • org.testng:testng ${testng.version} test
.github/workflows/cut_release.yml actions
  • actions/checkout v4 composite
  • actions/setup-java v3 composite
.github/workflows/run_release_tests.yml actions
  • actions/checkout v4 composite
  • actions/setup-java v3 composite
  • actions/setup-node v3 composite
validation_testing/Dockerfile docker
  • ubuntu latest build
athena-db2-as400/pom.xml maven
  • com.amazonaws:athena-jdbc 2022.47.1
  • net.sf.jt400:jt400 20.0.3
  • com.amazonaws:athena-federation-integ-test 2022.47.1 test
  • com.amazonaws:athena-jdbc 2022.47.1 test
  • com.amazonaws:aws-java-sdk-rds ${aws-sdk.version} test
  • org.mockito:mockito-core ${mockito.version} test
  • org.mockito:mockito-inline ${mockito.version} test
  • software.amazon.awscdk:rds ${aws-cdk.version} test
athena-gcs/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.47.1
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • com.google.cloud:google-cloud-storage 2.27.1
  • net.java.dev.jna:jna-platform 5.13.0
  • org.apache.arrow:arrow-dataset ${apache.arrow.version}
  • org.apache.arrow:arrow-memory-netty ${apache.arrow.version}
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j2-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j ${slf4j-log4j.version}
  • org.slf4j:slf4j-api ${slf4j-log4j.version}
  • com.amazonaws:athena-federation-integ-test 2022.47.1 test
  • org.junit.support:testng-engine 1.0.4 test
  • org.mockito:mockito-core ${mockito.version} test
  • org.mockito:mockito-inline ${mockito.version} test
  • org.testng:testng ${testng.version} test
  • software.amazon.awscdk:logs ${aws-cdk.version} test
athena-kafka/pom.xml maven
  • com.amazonaws:aws-athena-federation-sdk 2022.47.1
  • com.amazonaws:aws-java-sdk-glue ${aws-sdk.version}
  • com.amazonaws:aws-java-sdk-sts ${aws-sdk.version}
  • com.amazonaws:aws-lambda-java-log4j2 ${aws.lambda-java-log4j2.version}
  • com.fasterxml.jackson.core:jackson-annotations ${fasterxml.jackson.version}
  • com.google.guava:guava ${guava.version}
  • org.apache.kafka:kafka-clients 3.5.1
  • org.apache.logging.log4j:log4j-api ${log4j2Version}
  • org.apache.logging.log4j:log4j-core ${log4j2Version}
  • org.apache.logging.log4j:log4j-slf4j2-impl ${log4j2Version}
  • org.slf4j:jcl-over-slf4j ${slf4j-log4j.version}
  • org.slf4j:slf4j-api ${slf4j-log4j.version}
  • com.amazonaws:athena-federation-integ-test 2022.47.1 test
  • org.junit.support:testng-engine 1.0.4 test
  • org.mockito:mockito-core ${mockito.version} test
  • org.mockito:mockito-inline ${mockito.version} test
  • org.slf4j:slf4j-simple ${slf4j-log4j.version} test
  • org.testng:testng ${testng.version} test
validation_testing/cdk_federation_infra_provisioning/app/package.json npm
  • @types/jest ^29.4.0 development
  • @types/node 18.11.18 development
  • aws-cdk 2.63.2 development
  • jest ^29.4.1 development
  • ts-jest ^29.0.5 development
  • ts-node ^10.9.1 development
  • @aws-cdk/aws-glue-alpha 2.63.2-alpha.0
  • @aws-cdk/aws-redshift-alpha 2.63.2-alpha.0
  • aws-cdk-lib 2.63.2
  • dotenv ^16.0.3
  • source-map-support ^0.5.21
  • typescript 4.9.5