I have 2000+ test cases. Every time I run regression tests, I need to run a lot of tests. Currently, these tests cover the default functionality. However, we have a handful of customers (large companies) who want custom behaviors in different places of the code. These customers are grouped into subscriptions. So I need most tests to always test the same thing for every customer, but the custom behavior will likely require at least a couple hundred more tests for each subscription.
It's important to note that the Robot files are merely calling Python methods...The Python methods contain all test logic.
I could copy all 2000 robot files into separate folders (one for each subscription) and modify the tests that need to be modified, but that would not be maintainable.
I could tag the core tests that are applied to all subscriptions and then separate which tests are different for each subscription, but even that isn't manageable because I still have to manage which tests are core and which ones are not, and I don't want to move one from core to a custom test but then end up forgetting to add custom tests for all the other subscriptions that do not want that custom behavior. Nor do I want a test in the core group to accidentally be left there when a custom version of that test is created for a subscription.
So I landed on the following solution, but I need some help as to how to accomplish one part of it.
I will not create a single new robot file for any tests that test custom behavior. For the custom subscriptions, the existing corresponding tests will do different things based on what subscription number is passed to the test. Specifically:
if subscription == 1:
set TEST_ID to "TEST_ID=12345"
else if subscription is 2:
set TEST_ID to "TEST_ID=2345"
else:
set TEST_ID to "TEST_ID=01234"
Here is what I came up with, but there is a problem with it. (See below.) I should mention that for the sake of simplicity, I took out any references to Python code and replaced them with Log statements.
*** Variables ***
${SUBSCRIPTION_1} ${1}
*** Test Cases ***
This is the test case
${TEST_ID_TAG} = Set Variable If ${SUBSCRIPTION} == ${SUBSCRIPTION_1} TEST_ID=1234 TEST_ID=2345
[Tags] ${TEST_ID_TAG} TAG1 TAG2 TAG3_${SUBSCRIPTION}
IF ${SUBSCRIPTION} == ${SUBSCRIPTION_1}
Log python method 1
ELSE
Log python method 2
END
The problem is that the resulting tag is ${TEST_ID_TAG}
instead of "1". Strangely enough, TAG3_${SUBSCRIPTION}
does resolve to TAG3_1
as expected. I have tried putting the variable declaration in the variables section, but the entire line of Robot code ends up as the tag value. I should mention that I am aware that dynamic tags will not work for telling the Robot framework which tests to run. I only want these tags for reporting purposes.
Note: I am using robot 7.1