Skip to main content

Test Cases

The described test cases below provider a starting point to map the different features and capabilities that needs to be tested when changes occurs.


This is not intended to be a final or complete list but a starting point to reason about the different use cases and how to test them

Assumptions

  1. You have Harmonix on AWS platform deployed and running.
  2. You have several accounts that are used or can be used to provision environments and applications.
  3. You have an identity provider configured with backstage(Okta, AD, etc.)

Test case, context and expected result

We define a context of a test with the letter "C". This will allow us to describe the existing state before we can execute a test case.

We also define test case with the letter "T" which describe the set of instructions to test a scenario on Harmonix on AWS.

Lastly we define the expected result with the letter "E", this will describe the expected assertion of a resulted test case.

Context

IDDescription
C01An entitled user logged-in to backstage
C02A provisioned ECS Environment(Single provider) is available
C03A provisioned EKS Environment(Single provider) is available
C04A provisioned Serverless Environment(Single provider) is available
C05A provisioned AWS Environment is available
C06A provisioned ECS application(Single environment) is available
C07A provisioned EKS application(Single environment) is available
C08A provisioned Serverless application(Single environment) is available
C09A provisioned shared RDS Resource is available
C10A provisioned shared S3 Resource is available
C11A binding between an application and resource exist
C12An application is deployed to more than one environment
C13A provisioned ECS Environment(Multi provider) is available
C14A provisioned EKS Environment(Multi provider) is available
C15A provisioned Serverless Environment(Multi provider) is available
C16A provisioned ECS application(Multi environment) is available
C17A provisioned EKS application(Multi environment) is available
C18A provisioned Serverless application(Multi environment) is available
C19A provisioned Terraform application/provider/resource
C20User chooses that a new VPC should be created
C21User chooses that an existing VPC ID should be used instead of creating a new VPC
C22A provisioned EKS Cluster is available
C23A provisioned EKS Cluster Kubectl Lambda is available

Expected result

IDDescription
E01Successfully provision an ECS provider
E02Successfully provision an EKS provider
E03Successfully provision an Serverless provider
E04Successfully provision an Environment
E05Successfully provision an ECS application(Single provider)
E06Successfully provision an EKS application(Single provider)
E07Successfully provision a Serverless application(Single provider)
E08Successfully start and stop an application
E09Successfully view application logs
E10Successfully view application audit table
E11Successfully set environment variables to an application
E12Successfully view cloud related application resources
E13Successfully bind a shared resource to an application
E14Successfully deploy an application to additional environments
E15Successfully toggle between deployed environments of an application
E16Successfully delete an application from an environment
E17Successfully delete an application from all environments
E18Successfully delete a shared resource
E19Successfully delete a provider
E20Successfully delete an environment
E21Successfully view provider details
E22Successfully unbind a shared resource to an application
E23A new VPC is created for the provider
E24An existing VPC ID is used by the provider
E25Successfully provision an RDS database as a shared resource
E26Successfully provision an S3 bucket as a shared resource
E27Successfully create EKS provider using existing EKS cluster and create new kubectl lambda
E28Successfully create EKS provider using existing EKS cluster and existing kubectl lambda

Test cases

Platform engineering

Environment Providers

IDContextDescriptionExpected Result
T001C01+C20Provision an ECS environment provider with serverless compute (Fargate)E01+E23
T002C01+C21Provision an ECS environment provider with serverless compute (Fargate)E01+E24
T003C01+C20Provision an ECS environment provider with EC2 computeE01+E23
T004C01+C20Provision an EKS environment provider with serverless compute (Fargate)E02+E23
T005C01+C21Provision an EKS environment provider with serverless compute (Fargate)E02+E24
T006C01Provision an EKS environment provider with managed nodesE02
T007C01Provision an EKS environment provider with private-only API accessE02
T008C01+C20Provision a Serverless environment providerE03+E23
T009C01+C21Provision a Serverless environment providerE03+E24
T010C01+(C02,C03,C04)View provider information card - on provider pageE21
T011C01+(C02,C03,C04)Delete provider - on provider management pageE19
T012C01+C22Provision an EKS environment provider and import an existing cluster and create kubectl lambdaE27
T013C01+C22+C23Provision an EKS environment provider and import an existing cluster and reuse existing kubectl lambdaE28

Environments

IDContextDescriptionExpected Result
T200C01Provision an AWS environment with a single providerE04
T201C01+C05View environment information card - on environment pageE21
T202C01+C05Delete environment - on environment management pageE21
T203C01+C05Add additional provider to an existing environment - on environment pageE19
T204C01+C05Delete a provider from an existing environment - on environment pageE19

Developer

Application Provisioning

IDContextDescriptionExpected Result
T400C01 + C02Provision a node.js application on Amazon ECS environment with a single providerE05
T401C01 + C02Provision a Java SpringBoot application on Amazon ECS environment with a single providerE05
T402C01 + C02Provision a python application on Amazon ECS environment with a single providerE05
T403C01 + C02Provision a node.js Terraform application on Amazon ECS environment with a single providerE05
T404C01 + C03Provision a node.js application on Amazon EKS environment with a single providerE06
T405C01 + C03Provision a Java SpringBoot application on Amazon EKS environment with a single providerE06
T406C01 + C03Provision a python application on Amazon EKS environment with a single providerE06
T407C01 + C03Provision a node.js Terraform application on Amazon EKS environment with a single providerE06
T408C01 + C04Provision a Serverless REST API application on AWS Serverless environment with a single providerE07

Application operations

IDContextDescriptionExpected Result
T600C06Start and Stop the applicationE08
T601C06View application cloudwatch logs - on App logs tabE09
T602C06View application audit table - on audit tabE10
T603C06Add/edit/delete environment variables for the applicationE11
T604C06View related cloud resources for the applicationE12
T605C06 + C02Deploy application to another available environmentE14
T606C06 + C12Toggle between deployed environments for the applicationE15
T607C06Delete an application from an environmentE16
T608C06 + C12Delete an application from all environmentsE17
T609C06 + C09Bind a shared resource RDS to an applicationE13
T610C06 + C09 + C11Unbind a shared resource from an applicationE22
T611C07Start and Stop the applicationE08
T612C07View application cloudwatch logs - on App logs tabE09
T613C07View application audit table - on audit tabE10
T614C07Add/edit/delete environment variables for the applicationE11
T615C07View related cloud resources for the applicationE12
T616C07 + C02Deploy application to another available environmentE14
T617C07 + C12Toggle between deployed environments for the applicationE15
T618C07Delete an application from an environmentE16
T619C07 + C12Delete an application from all environmentsE17
T620C07 + C09Bind a shared resource RDS to an applicationE13
T621C07 + C09 + C11Unbind a shared resource from an applicationE22

Shared resources

IDContextDescriptionExpected Result
T900C01+(C02,C03,C04)Create a new RDS database resourceE25
T901C01+(C02,C03,C04)Create a new S3 bucket resourceE26