SoftTree Technologies SoftTree Technologies
Technical Support Forums
RegisterSearchFAQMemberlistUsergroupsLog in
[10.1.278 Pro] - Unit Test window anomalies

 
Reply to topic    SoftTree Technologies Forum Index » SQL Assistant View previous topic
View next topic
[10.1.278 Pro] - Unit Test window anomalies
Author Message
gemisigo



Joined: 11 Mar 2010
Posts: 1592

Post [10.1.278 Pro] - Unit Test window anomalies Reply with quote
I've run into several issues while using the Unit Test feature.

#1. The selection window does not retain its settings (name matching method, do not show empty schemas, etc.). As soon as the objects are selected, those settings are gone, reopening the selector starts with the defaults.
#2. Filtering has no effect when checking. Though it helps a bit to only see the filtered object, there's no convenient way to select those, you have to do it by checking each of them one by one. Checking the procedure will select all the procedures regardless of the filter and that's not what one would expect. For example, the popup filter for columns does not work that way.

See this short video here for #1 and #2.

#3. Changing the order of the Test Units makes the overlay icons indicating success/failure inoperable for those cases. They won't work again until the windows in closed, reopened and the project reloaded. Again, a short video about the issue here

Watch this short video here for #3.

#4. Selecting a couple of objects (tables, views, stored procedures, whatever) create the Unit Tests and their respective Test Cases and gives them names. For Unit Tests, this name might even be helpful but the Test cases only get generic names such as SELECT for views, SELECT, INSERT, UPDATE, DELETE for tables, and EXECUTE for stored procedures. Those are pretty much useless when it comes to the problem of determining which of Test Cases have actually failed.

See short video here for #4.

Could you add the object names to the Test Cases, please? That would make them much easier to use.
Fri May 24, 2019 5:37 pm View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6950

Post Reply with quote
Thank you. I was able to reproduce all three issues and submitted them as defects in our internal bug tracking system. I hope they are minor and can be fixed soon in time for the next release.
Sun May 26, 2019 6:27 am View user's profile Send private message
gemisigo



Joined: 11 Mar 2010
Posts: 1592

Post Reply with quote
That's great.

Any opinions on #4?

Manually editing those names in the Unit Test window is really a pain in the back. So much so that I've developed a habit of editing .unittest xml files directly in a text editor instead of the Unit Test window. Sadly, that has the drawback of having to constantly reload the file (which process is slow on its own now).

There's also no way to move Test Cases between Tests other than editing the XML.
Tue Nov 05, 2019 10:10 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6950

Post Reply with quote
Sorry for the late reply. You can rename all test cases and unit test as you see the fit. The generated names are simply default names to start with. Is that something that would work for you?


Wed Nov 06, 2019 3:05 am View user's profile Send private message
gemisigo



Joined: 11 Mar 2010
Posts: 1592

Post Reply with quote
SysOp wrote:
Sorry for the late reply. You can rename all test cases and unit test as you see the fit. The generated names are simply default names to start with. Is that something that would work for you?

I am aware of the ability to rename Test Cases and Unit Tests. And no, sadly, that doesn't work well. That is what I referred to as "a pain in the back". It's slow. The whole way of how Unit Test window works is quite a bit less helpful than it could be.

I'm not sure how others use this but I don't create tests to test the structure, that would be redundant. I seldom write code to create tables, views, etc. I design the database objects in a designer and then generate the code and creating the default Unit Tests + Test Cases would only test the "correctness" and "completeness" of the designer tool I use. And that's not something I want or need. I trust that tool (wouldn't use it otherwise).

On the other hand, I do business logic in stored procedures required by features with that gray mush in my skull. What I need Unit Tests for is to test features and test those stored procedures.

Each test will contain multiple test cases. Now, if I know that the feature I want to test uses, let's say, 20 tables, I could choose two paths:
#1 create a test with a single case that uses a test data generator project and add generic Test Cases to test the feature itself or
#2 drop 20 test cases that each creates test data for its own table which I can edit in-place which can be done quickly without having to change the generator project in another editor (which is slow). I could then also test a few things on the fly (triggers on that table or whatever) which is not possible with #1. And at last, again, add some generic tests where I test the feature itself.

Now, I like #2 a bit better as it provides greater control overall. But, you see, #2 (separate test cases for the 20 tables) is also quite a bit of a nuisance, because:
a) I either have to write each Test Case from scratch (no pre-generated INSERT/SELECT/DELETE/UPDATE statements for the tables) which slows the whole process down and is error-prone, or
b) I select the tables with New Table-based Unit Test, where all the tables will end up as Test Cases in their own separate Unit Tests instead of putting them into one Unit Test and it's hard to move the Test Cases (though can move them using Ctrl+Up/Down, drag&drop doesn't work, and clicking the icons several dozen times feels, well, "outdated")
To make it worse, there's no way to move the Test Case between the Unit Tests in the Unit Testing window.

The only way to kick them around is to manually edit the xml. This also slows the process down, plus I have to deal with those hateful generic names. When running a Unit Test that touches 20 tables, having to count the lines by poking my finger at the screen to get which of the 20 INSERT Test Cases was the one returning with "Test case "INSERT" failed" gets annoying after the first try. So I have to rename each of the INSERT/SELECT/DELETE/UPDATE Test Cases generated with the default name, which means a couple hundred (or thousand) renames in the long run for even a medium-size feature set. If those generic names could be, eg. INSERT schema_name.table_name instead of simply INSERT, that would have spared me hours (days so far) of tediously eroding the surface of my keyboard so far.

The Unit Testing feature of SQL Assistant has great potential, but in its current state (no Save, just Save As, cannot move Cases between Units, having to edit in an external editor and rename almost every Test Case, no multi-selection of Units/Cases, can only delete one at a time, etc.) it is severely restrained and utterly uncomfortable to use. Not that I know any better alternatives, of course, so I am and will be using it, but it hurts user experience.
Wed Nov 06, 2019 6:38 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6950

Post Reply with quote
You can already create custom unit tests with a custom set of test cases, using macros to automate their code to a great extend. In my humble option supporting macros in unit test names and test case names would meet your requirements. It would enable you to create your own naming conventions.

I'm going to ask about the feasibility of that.
Wed Nov 06, 2019 9:38 am View user's profile Send private message
gemisigo



Joined: 11 Mar 2010
Posts: 1592

Post Reply with quote
SysOp wrote:
You can already create custom unit tests with a custom set of test cases, using macros to automate their code to a great extend.

Yes, I'm already doing that. If you check the first screenshot you can see that it has a New Function-base(-d) Unit Test. I added that one because originally the procedure-based was supposed to handle both the procedures and functions but that template had the following code in its execute part:
Code:

CALL $OBJECT$ (
   $ARGS_V$=?
);

and while you can CALL a procedure, you have to SELECT a function. I've also modified the original for the procedure to:
Code:

CALL $OBJECT(ins_object, proc)$ (
   /*{$ARGS_V$=?}*/
);

because passing named arguments to a procedure is not allowed in MySQL/MariaDB. I've also added ins_object (to remove the schema name) and proc in hope that it will filter out functions, alas, that didn't work. Then I saw that the Unit Test has its own property named Object Type, which is set to Procedure/Function and simultaneously realized that the Select Objects dialog is not a popup, so the filtering is not done by the macro.

SysOp wrote:

In my humble option supporting macros in unit test names and test case names would meet your requirements. It would enable you to create your own naming conventions.

I'm going to ask about the feasibility of that.

Very good idea. Thank you very much! That would be great. Even a limited set of macros would be a tremendous help (I imagine $OBJECT$ alone could do the job).
Wed Nov 06, 2019 10:55 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6950

Post Reply with quote
I just received email confirmation that supporting $DB$, $SCHEMA$, and $OBJECT$ macros in names is a very easy task, and we are going to add that in time for 11.1 release.
Wed Nov 06, 2019 11:44 am View user's profile Send private message
gemisigo



Joined: 11 Mar 2010
Posts: 1592

Post Reply with quote
Brilliant! Eagerly awaiting 11.1 then.
Thu Nov 07, 2019 4:03 am View user's profile Send private message
Display posts from previous:    
Reply to topic    SoftTree Technologies Forum Index » SQL Assistant All times are GMT - 4 Hours
Page 1 of 1

 
Jump to: 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


 

 

Powered by phpBB © 2001, 2005 phpBB Group
Design by Freestyle XL / Flowers Online.