It's a good idea, but you'd ultimately have to trust the project to tell the truth about whether they really are free software or not.

That sounds simple but, especially around the edges, people can disagree about what is free and non-free. It's quite common for free software projects to include some non-free components, either intentionally or unintentionally.

For instance, a project that doesn't really know or care about freedom could advertise itself as open source, but include blobs, or use a non-standard license that doesn't actually give you the four freedoms.

Even well-meaning projects can accidentally include incompatibly licensed code, or code that they actually don't have permission to distribute to you, by accident.

A good heuristic, for Linux, is that if it's packaged in Debian (ignoring contrib, non-free or non-free-firmware) or Fedora's archives, it's probably free software. This is because those communities really care about freedom, vet the packages for licenses and check that the four freedoms are actually given. Things do sometimes slip through, but when they are found, the packages are fixed or removed.

For Android, if it's on F-Droid, it's almost certainly free software, for the same reason.

Meanwhile, the FSF have a free software directory that contains a listing of programs they consider to be free.