You can find crimson stems in the crimson forest biomes, from huge fungi. They can be picked up by breaking them with any tool but using an ax is the quickest. Crimson stems cannot be found in any naturally generated structures or in loot chests.
Huge fungi found in the crimson forest biome (via screenshot)
ncG1vNJzZmivmaC2b7%2FPqKmtq5uasqWtjZympmednrumr9Gana1nk6e2rr%2FOp2SsrJWieqO4zpyiZqGeYrqqusScqZqepA%3D%3D