Getting Started with QA testing

Hi there! I’d like to help with QA testing and have so far:

Next, I plan to:

Is there anything I am leaving out?

I have 1 question about editing the wiki to update test results - I can’t login to wiki because it says “You need to have at least CLA+1 (group)” to login. How can I get this fixed or does relval/easy karma take care of this for me?

Thanks (in advance) for any advice or info!

ps. I saw your post @FranciscoD , but I couldn’t follow up or reply on it! :smile:

3 Likes

Hello @dan1mal!
Nice to know you want to join QA. Testing updates and upcoming releases is a very important task, and it could be funny and exciting. And more people test things, the better.

Since you already introduced yourself to the mailing list, someone should sponsor you in the qa and in the fedorabugs FAS groups. As far as I can see you are now in these groups, so you are CLA+1. Maybe in the meanwhile somebody has approved your membership.
Can you retry to edit the wiki?

Happy testing!

2 Likes

Ah, yes—we closed that post to limit it to the announcement :slight_smile:
Welcome to QA! :bug:

1 Like

Oh yes! I saw that sponsor email from adamwill, and I just tried to login to the wiki and was able to! Thank you @alciregi.

I’ve been scanning the Rawhide tests and noticed that a lot of tests are done by coconut/automated and many of the other ‘standard’ tests on regular hardware have been done by others. So, I had a question. As a new tester, should we focus on the out-of-norm cases or is there an alternative way to proceed?

In terms of my vm, I’ve downloaded Oracle virtualbox and installed. I also have another HDD (from an old laptop) that I have loaded with Silverblue. There’s still partition space for another installation and I was thinking of using it for a testing ground. Although, I’m still researching whether there’s a functional difference between testing on the bare metal or in virtualbox.

1 Like

Thanks FranciscoD! Glad to be here :smiley:

Whatever you like really. There’s no right way. The suggested way is to start with simpler tasks (like testing packages in updates-testing) and then joining more complex test days like kernel etc. The suggested way, however, is for folks who are completely new to testing since it takes them through the steps one by one. If you are technically good already, and have experience with such things, feel free to dive in to the deep end :clap:

Yes, talking about rawhide/upcoming release, you can test what you want. More people run test cases, the better. Maybe something that works on your hardware/setup, doesn’t work on my hardware/environment. This is also true for automated test, even if they are pretty reliable (if I’m not wrong, these automated test are performed in KVM virtual machines).

Testing things in Virtualbox is good, since Oracle Virtualbox is a widespread system, so if something doesn’t work here, it is better to catch bugs as soon as possible.
Obviously, baremetal systems out there are most diverse. Again, more model tested, the better.

I suggest that if you have doubts or you need deeper assistance, you should ask your questions in the QA team mailing list.

Thanks.

1 Like

I’d like to stress again (though @alciregi sort of said it) that there’s definitely a functional difference between testing in a VM and testing on a baremetal. Bug specific to your hardware won’t be seen in a VM as VM presents to OS a specific subset of emulated hardware, by default it won’t passthrough your actual hardware to the guest OS.

As far as I understand it, if you and me use VirtualBox VM for testing on a widely different hardware, we’re very likely to see the same bugs in the VM. And on a baremetal we’re likely to see different subsets of bugs – due to differences in our hardware.

Using VM is simple and safer though )

3 Likes

Yep.

Well, in a VM you can still catch at least two types of bugs:

  • bugs related to running the OS in a VirtualBox (or any other hypervisor) VM
  • bugs not related to hardware, but for instance problems related to specific softwares or related to the DE, etc.
2 Likes

So basically it’s useful to do both, as it allows to catch different types of bugs in different environments – as well as bugs common to all of these environments.

But of course, if you can’t do both – choose one way you’re more comfortable with. Though, I think, baremetal can be more useful – as we have more diverse environments for different users/testers here.

2 Likes

Well, I agree, but…
If you haven’t a baremetal machine to dedicate to tests, it is strongly discouraged to perform tests on your production machine.
However it depends which test you want to perform. For instance you can enable the updates-testing repository just to test new software (i.e. LibreOffice, Evince, etc.), but avoid kernel or other core packages updates.

2 Likes

Do you think a separate HDD on a production machine, then changing boot drives on startup be okay for baremetal testing?

Also, when you enable updates-testing repositories, and download the software, do you just open it up and play around inside it to see if anything unusual happens?

1 Like

Yes, if you have resources and time, you can do it for sure.

Yes, I use it normally in my daily work, and I look if it works as it was working before.

1 Like

Also you need to understand, that there’s a chance – quite small but still – of OS on another boot device affecting your primary OS on a production machine.

Simplest example would be an update to grub corrupting boot configuration on a primary drive, making you unable to boot back to you production OS. This exact thing should be quite easy to correct though. I can’t come up with another example from the top of my head. )

I don’t think the risk is too great, you just need to know it exists – so that you won’t be caught off guard if something happens, and so you wouldn’t say no, I don’t play with this anymore.

Also I was pointed some time ago that for some packages there’s a recommended tests to be run. They would be listed on a Bodhi page for such an update / package, if there’re any. I don’t have an example ready.

2 Likes

:+1: Yup.
For instance, Firefox: https://bodhi.fedoraproject.org/updates/FEDORA-2019-a1b709fca0 (this update is already in the stable repository) has various test cases. You can find them at the bottom of the page, or in the Test Cases tab.

3 Likes