This is a question that has come up surprisingly often. And while the answer is pretty straightforward, it's only obvious if you know there's more than one way to power on a GoPro.
$50 off the HERO7 Black at GoPro.com
You can get the HERO7 Black directly from GoPro.com at $50 off, which brings it down to $349.99.
But you'll have to be quick--it ends March 23. They're also throwing in an SD card and free 2-day shipping and returns.
You can find the deal here.
That's because many GoPros actually have two ways to turn the camera on. One simply powers the camera on. The other turns the camera and and immediately begins shooting. It uses a feature call QuikCapture (and no, that's not a typo--it's how GoPro spells it).
So if you're asking this question, it's almost certainly because you're turning the camera on with the shutter button (otherwise known as the large button with the red circle on top). If you have QuikCapture enabled, that's how you start up in QuikCapture mode. You have some control over which shooting mode it starts in as well as being able to turn the feature off; I have more details on that here.
But if you want to just turn the GoPro on without recording, use the other power button. Where that is varies by GoPro model.
On the newer, larger-style GoPros with the integrated waterproof housing, it's a gray button on the side.
On the smaller Session-style GoPros, it's a small black button on the bottom of the back of the camera.
On the HERO4/3 models, it's the large button on the front.
Those are the main power buttons, and they'll turn the camera on without starting the recording immediately.