Both patterns come for the same necessity: Hide from some client code the construction logic of a complex object. But what makes "complex" (or, sometimes, complicate) an object? Mainly, it's due to dependencies, or rather the state of an object composed by more partial states. You can inject dependencies by constructor to set the initial object state, but an object may require a lot of them, some will be in a default initial state (just because we should have learned that set a default dependency to null is not the cleanest way) and some other set to a state driven by some condition. Moreover, there are object properties that are some kind of "oblivious dependencies" but also they can assume optional states.
there are two well known ways to dominate that complexity:
Composition/aggregation: Construct an object, construct its dependent objects, then wire together. Here, a builder can make transparent and flexible the process that determines the rules that lead the construction of component.
Polymorphism: Construction rules are declared directly into subtype definition, so you have a set of rules for each subtype and some condition decides which one among these set of rules apply to construct the object. A factory fits perfectly in this scenario.
Nothing prevents to mix these two approaches. A family of product could abstract object creation done with a builder, a builder could use factories to determine which component object instantiate.