Animated GIFs aren't just about memes. They can also be very handy for illustrating actions on a screen for a software tutorial or for comparing before and after shots. (I use Screenflow for creating them, but there are a lot of good apps that can also create screencast GIFs.)
But whether you're trying to embed a gif of the pop culture meme du jour or a technical how-to guide, there's a simple trick in WordPress to get your animated GIF to work.
The problem comes down to what WordPress does with images when you upload them to the Media Library. When you hit the Upload button, WordPress adds the image to its database and creates different-sized thumbnails for use in various parts of the site. Those are the ones that you can find in the Media Settings panel or any custom ones you've set up.
But the resized thumbnails are always JPG still images. If you upload a GIF, WordPress will create resized JPG thumbnails, not resized GIF thumbnails. So if you embed one of those resized versions in your post you'll end up with, well, a still image.
All you need to do to fix it is to make sure that you don't use one of the resized thumbnails but instead use the original GIF file you uploaded. In WordPress, that's known as the "Full Size" version. So when you're choosing the image to embed, go to the options to the bottom right and make sure to choose "Full Size" from the list.
Your list might look a bit different--I've added some custom thumbnails sizes. But the Full Size option is baked into WordPress.
It's one of those things that's incredibly simple when you know it but can create frustration if you don't.
Using a Plugin
While it's easy enough to embed GIFs without a plugin, if you prefer to use a plugin to insert GIFs from the Giphy service, you can. The most popular one is GiphyPress.
It adds a new button to the post editor and helps you discover new GIFs to use from the Giphy website. But it's tied in tightly to their site, and it's not designed for directly uploading your own GIFs like the screencast GIF above.