It would really help to know what concepts you are already familiar with and feel like you understand. Edit: it would also be helpful to know what you are trying to do that has you working with ports or using a VPN. There are countless situations where ports and VPNs are involved. /Edit Lacking that, I'm going to cover a lot of basic concepts. I'm assuming typical/common situations, as always it can always be different and more complicated. Wall of text incoming.
Typical basic networks are a collection of devices that are able to communicate with each other through some medium like a wired network or WiFi. Each device In a network will have an IP address (a set of four numbers between 1 and 254 divided by periods for example: 192.168.1.100). For one device to send a message to another device, it addresses the message to the device using its IP Address. This is like sending a letter in the mail - you need to mark the letter with the street address or it won't be delivered.
Now sticking with this analogy, if I wanted to send you a letter and you lived in a hotel or apartment complex without a dedicated address just for you, I need to make sure to address the letter not just to your building but also to you specifically. This is where ports come in. If I want to join your Minecraft server, I need to be able to reach your computer specifically and I also need to make sure my Minecraft game is talking to your Minecraft server and not your web browser or something. Software listens for specific ports. So to join a Minecraft server hosted on your computer with the address 192.168.1.100 I'd need to also specify the port so that the Minecraft server software know my messages are for it specifically.
That's the ELI5 version.
Stepping up the complexity now. In most cases, joining someone's Minecraft server is more complicated because I'm on my home network, and my router is connected to the internet, and through the internet I connect to your router and then from your router I need to connect to your computer. The way this works is that each of our routers are on two different networks - our home network and the Internet.
Most home networks will look very similar. There are ranges of addresses that are reserved for local (non internet) network devices, such as my example IP address - 192.168.1.100. It is possible that each of our networks (and countless other home networks) have a device with this IP address so I can't just send a message to 192.168.1.100 and have it get to your networks 192.168.1.100 address. Fortunately, routers act as middlemen and coordinate communication between devices inside our network and devices on the internet. Think of the router as a fancy doorman outside the hotel or apartment who knows what room all of those living there are staying.
Unfortunately, it is often the case that my computer will not know the IP address of your computer within your network. You can work around this by using port forwarding. I write "Minecraft" on the letter and you instruct your doorman that any letter with "Minecraft" written on it should be delivered to your Minecraft server. In reality, you'd forward any traffic from the internet with a specific port number directly to your Minecraft server computer. The computer running the server is already "listening" for traffic on that port and will take it from there.
Now for more complicated stuff. Technically, ports are rather arbitrary. Some ports are standardized. As someone else had mentioned, port 80 is the standard for HTTP and 443 is standard for HTTPS (encrypted). However, nothing is stopping you from hosting a webserver on a server using a different port, like port 12345. Your web browser will assume port 80 and the webserver will ignore your web browser until you tell your browser to use port 12345. It is also possible for multiple service or pieces of software to have the same port which causes problems. I have a server on my home network and multiple services that display a web page (port 80 or 443). They can't all use the same port so I have to work around this by either using different ports or assigning each a different IP address.
So what does "opening" a port mean? Well, as you might imagine there are many pieces of software and many services running on devices in your home network. As a security precaution, many routers will block or ignore any traffic on the internet that tries to talk to it in a way it is not expecting. You don't want randos trying to connect to your computer through the internet and your router.
Unblocking or opening a port could be something like forwarding a port to a device on your network. Technically, this would involve finding a "Port Forwarding" setting on your router. Here, you would indicate any traffic with port 25565 (the default Minecraft server port) should be directed to your server at address 192.168.1.100. Now your doorman will always deliver every letter with "Minecraft" written on it to your Minecraft server. Be aware that even if you aren't the server software, every letter like that will continue to be directed to your server. If there is a security flaw in Minecraft, this could technically be something someone could exploit.
Similar to your router, Windows (the operating system) will also block/ignore traffic that it isn't expecting as a security precaution. Opening a port might mean creating a rule in your firewall setting to let traffic from certain ports into your computer. If Windows is ignoring any letters it doesn't recognize, my Minecraft letters still won't reach your Minecraft server even if it is clearly addressed to it. Windows may also be blocking traffic from leaving your computer if it does not have a firewall rule telling it otherwise so its possible the server got my letter, but Windows blocked the reply so I'm left hanging.
A VPN is a Virtual Private Network. This term can mean a few different scenarios but what they all have in common is that it creates a hole into your network, bypassing the router (maybe both routers). It is sometimes called a tunnel, which is a pretty good metaphor. Basically, you are running a virtual network cable through the internet from one computer or network to another. Through this virtual network, devices will be able to see and interact with each other as if they were on the same network.
Real life example. I manage a few servers where I work. If I need to log into those servers from home I cannot do this. It would be extremely dangerous to just let anyone on the internet have access to the server's login screen. So instead, I use a VPN. Now my home computer is virtually connected to my work's internal network - as if I'd brought my personal laptop into work and plugged it into the network jack next to my workstation in my office. I can use the same software I'd use at work to log into my servers without having those servers directly exposed to the internet. I am limited by my home internet speeds but it is otherwise almost identical.
Similarly, we could install VPN software on our two computers and I could then join your Minecraft server directly, bypassing your router, as if we were both on your home network.
Like I said, there are quite a few ways to do this and I've heard many of the ways referred to as either a VPN or a tunnel. More context could get you a better answer.
Generally, opening ports is a security risk. This is especially true when opening more common ports or ports that get a lot of use. More common, standard ports will be more likely to get attacked and ports that get a lot of use will be more likely to be targets of exploits. It really depends on your scenario. Imagine that, if someone wanted, they could likely figure out what port you've opened and what is listening for that port on your home network. They might try to brute force a password or take advantage of the software's security flaws.
Forwarding a randomly chosen port so a friend can join your Minecraft server? Not so risky. Opening port 22 so you can log into your home computer while you are away from home? Much riskier. Using an atypical port for remote access is still risky as, like I said, you have to assume that if someone really wanted to, they could figure out that remote access is running on port 17426 or whatever random number you may have chosen.