In an ever-expanding landscape called Internet of Things and the exploding development of artificial intelligence, we are bombarded with complex methods of integrating data science into the software development process – all in the name of ensuring efficiency in our operations and coming up with the best output.
Admittedly, some processes are little known to us until we realize the need for interdependence, collaboration, and sharing — that is, bringing the different elements of data science and software development into a relationship.
Because we’ve been there and we’re still at it improving our ways, believe us when we say that these three could very well be the tricks of the trade.
1. Rethink and Restructure Data Science
Think of rebooting your knowledge about data science. This time, define and state your goals and purpose by providing value to the core offering of your product/service. The initial goal for data science is to work with product management and software development on features that enhance your current offering.
Especially for startups, setting your goals at the get go – even before a team is formed, provides compass and focus – stick to the plan! You don’t want to get sidetracked and derailed along the way.
To keep your goals in track, establish a reporting structure. Here, it should be emphasized that the Engineering and Product teams should agree to collaborate. One is never superior to another. The structure of the game is rather simple — to work as peers. Data science working closely with engineering will provide opportunities for the former to grow into a research and innovation arm for the company.
2. Reinforce the Role of Data Science in Systems Development Life Cycle and Appoint a Point Person
With data science and engineering working hand in hand, know how data science takes part in the cycle. Since data science holds the stakes for data science related features, the team has a say in the innovative features integrated into the product/service. They also help in QA and product management in putting approval stamp to the work.
A great part in the efficiency of collaboration is how the work is being communicated. You don’t want your data scientists and engineers to speak in different languages and pace in a totally different mode. This principle can also help the teams be more open to others’ perspectives and advice.
It will do well to appoint a point person — an expert developer within the engineering team who can understand the “data science-speak” and is well-versed in translating feature requests into actual code. The point person will assist in making sure that the instructions are passed and clarifications are communicated. It is expected of the point person to anticipate potential problems and communicate complex information to the developers.
3. Develop Work Methodology and Use the Right Tools
The field of data science is a challenging lot, let alone the nitty-gritty of work can be overwhelming. It will serve you well to raise your awareness by developing work methodology and setting timelines. Think about strategies and tools you would use to lighten the load.
Conversely, it is also important that you determine what doesn’t work. For example, data science on Kanban can set artificial time-blocks which simply might not work to your favor because of the dynamic nature of work.
In your tracking tool of choice, complex work should be plotted into small achievable tasks. You don’t want your team to get stuck with a heavy load or juggling multi-task seeing not a step completed by the end of the day.
This leads to how you use available tools. Get a sense of the bigger picture – how complex the work of data science is and the role tools play in it. Do not expect cloud-based software developed by your team to take care of the whole lot, such as loading, processing, prototyping, customizing, analyzing, and reporting — this can translate to a massive volume of data that can be impossible to handle.
Consider this one quick, practical suggestion: Have data science maintain its own code repository and shared libraries hosted on a private PyPi server. Moreover, utilize tool libraries maintained by members of the data science team who are knowledgeable in Python. These libraries can also serve as prototypes for features that could be added to the platform by engineering.
Photo courtesy of gdsteam.
