We introduce the first version of GPU4PySCF, a module that provides GPU acceleration of methods in PySCF. As a core functionality, this provides a GPU implementation of two-electron repulsion integrals (ERIs) for contracted basis sets comprising up to functions using the Rys quadrature. As an illustration of how this can accelerate a quantum chemistry workflow, we describe how to use the ERIs efficiently in the integral-direct Hartree-Fock build and nuclear gradient construction. Benchmark calculations show a significant speedup of 2 orders of magnitude with respect to the multithreaded CPU Hartree-Fock code of PySCF and the performance comparable to other open-source GPU-accelerated quantum chemical packages, including GAMESS and QUICK, on a single NVIDIA A100 GPU.