This paper proposes an efficient approach for the extraction of the brain evoked potentials (EPs) from a small number of sweeps. In this approach, the ensemble average EP waveforms of different electrodes are computed from a small number of sweeps. Theses EP waveforms are decomposed into uncorrelated components using a correlation-based blind source separation method. By this separation, the evoked potential waveforms could be isolated from the electroencephalogram (EEG), noise and other artifacts waveforms. In order to enhance the signal-to-noise ratio and to recover the desired EP components, each uncorrelated component is filtered through a zero-phase matched filter based on the third-order correlation lags of the filter input. Finally, the evoked potential waveform recorded by every electrode is obtained through the projection of the selected filtered evoked potential components into the electrode space. Experimental results for visual and auditory evoked potentials show that the EP waveforms obtained by applying the proposed approach to 5 sweeps for the visual case and to 2 sweeps for the auditory case are better than those obtained by ensemble averaging and filtering of 45 sweeps.