ADP-ribosylation factors (ARFs) and their activating guanine nucleotide exchange factors (GEFs) play key roles in membrane traffic and signaling. All ARF GEFs share a ∼200-residue Sec7 domain (Sec7d) that alone catalyzes the GDP to GTP exchange that activates ARF. We determined the crystal structure of human BIG2 Sec7d. A C-terminal loop immediately following helix J (loop>J) was predicted to form contacts with helix H and the switch I region of the cognate ARF, suggesting that loop>J may participate in the catalytic reaction. Indeed, we identified multiple alanine substitutions within loop>J of the full length and/or Sec7d of two large brefeldin A-sensitive GEFs (GBF1 and BIG2) and one small brefeldin A-resistant GEF (ARNO) that abrogated binding of ARF and a single alanine substitution that allowed ARF binding but inhibited GDP to GTP exchange. Loop>J sequences are highly conserved, suggesting that loop>J plays a crucial role in the catalytic activity of all ARF GEFs. Using GEF mutants unable to bind ARF, we showed that GEFs associate with membranes independently of ARF and catalyze ARF activation in vivo only when membrane-associated. Our structural, cell biological, and biochemical findings identify loop>J as a key regulatory motif essential for ARF binding and GDP to GTP exchange by GEFs and provide evidence for the requirement of membrane association during GEF activity.