An Empirical Study on Python Library Dependency and Conflict Issues

With the rapid development of open-source communities, code reuse in Python projects is increasingly common. Developers heavily rely on third-party libraries from the Python central repository. They need to write specific configuration scripts with version constraints to ensure the correct version of dependent libraries when building projects. However, existing research focuses on direct dependency libraries and ignores potential dependencies that may exist in other dependency configuration files (e.g., requirements-dev.txt for development dependencies). To fill this gap, we conduct an in-depth comprehensive study to quantify the distribution of direct and potential dependencies, correlation, and classification along with detection tools of dependency conflict issues with 278 top popular Python library projects, which were collected from the prominent dependency tracking system Libraries.io. Specifically, we first investigate the magnitude distribution of dependencies by parsing library source files. Second, we visualize dependencies among Python libraries to determine the correlation. We then classify types of dependency conflicts from the perspective of third-party libraries. Finally, we compare Python dependency conflict detection and resolution tools for researchers and developers. Our findings show that third-party libraries containing dependencies are more common, with 79.1% having at least one dependent library. Moreover, the dependency relationship among Python libraries is intricate, which generates lots of dependency conflict issues such as version conflicts. The main cause of issues is the conflict among third-party libraries, accounting for 60.13%. Our findings can help developers better understand library dependencies and provide them insights on how to better manage them.