This work proposes to use the radiation from brachytherapy sources to track their dwell positions in three-dimensional (3D) space. The prototype device uses a single flat panel detector and a BB tray. The BBs are arranged in a defined pattern. The shadow of the BBs on the flat panel is analyzed to derive the 3D coordinates of the illumination source, i.e., the dwell position of the brachytherapy source. A kilovoltage x-ray source located 3.3 m away was used to align the center BB with the center pixel on the flat panel detector. For a test plan of 11 dwell positions, with an Ir-192 high dose rate unit, one projection was taken for each dwell point, and locations of the BB shadows were manually identified on the projection images. The 3D coordinates for the 11 dwell positions were reconstructed based on two BBs. The distances between dwell points were compared with the expected values. The average difference was 0.07 cm with a standard deviation of 0.15 cm. With automated BB shadow recognition in the future, this technique possesses the potential of tracking the 3D trajectory and the dwell times of a brachytherapy source in real time, enabling real time source position verification.