The Nephilim were on  the Earth at that time (and also immediately afterward), when those divine beings were having sexual relations with those human women, who gave birth to children of them.
These children became the heroes and the legendary figures of ancient times.

Genesis 9_4, International Standard Version