Understanding Agile Development
Let me start by saying that agile development is fundamentally sound and has serious merit. It may in fact be the best development methodology there is. However, that does not mean there are no serious issues with it in practice. In this blog, I will point out some of the major shortcomings that I have seen in its practice from some of the most gifted developers that I’ve ever worked with.
One key area that often gets overlooked is the importance of master data modeling. My hope is to educate developers so that these shortcomings can be reduced if not eliminated.
Developer Approach to Application Code
Most developers exercise deliberate and prudent care when developing their application code classes. Some use diagramming tools, while the rest simply rely upon methodical design planning and review processes – often which are entirely manual. Regardless, application developers generally create very flexible and reusable classes.
Effective Communication and Understanding of Code
Moreover, they can explain their classes to new team members in a very concise and understandable fashion. I’ve never heard a senior developer when questioned about their classes say “I don’t know what that it’s for” or “you’d have to read the code to figure out what it’s for”.
In general, they both fundamentally and intrinsically value classes since that’s the key item they work with. Hence they just know what the classes are for because that’s where their focus is.
Lack of Clarity in Database Design Among Senior Developers
Uncertainty in Table Usage
- When asked about the purpose of various tables, responses from senior developers following agile methodologies reveal a surprising lack of clarity.
- Some admit outright: “I simply don’t know.”
- Others provide uncertain responses, such as “I think it’s for X, but it might also be for Y.”
Issues with Duplicate Tables
- Duplicate tables emerge due to individual preferences:
- Developer #1 uses one set of tables.
- Developer #2 relies on a different set.
- This leads to redundant data structures and increased complexity in understanding the data model.
Absence of a Comprehensive Data Model
- When requested to present a unified data model, none exists.
- Despite the use of diagramming tools to design classes, data modeling is neglected.
Data Storage as a Secondary Concern
- Persistent data storage (e.g., tables, key-value stores, wide column stores, document, or graph databases) is often viewed merely as a means to an end.
- This approach results in poor design and limited understanding among developers.
Importance of Observations and Issues in Product Management
Why Observations Matter
As a product manager, when I perform my final user acceptance testing of a product before release, I tend to find major product issues that almost invariably are data-related.
Challenges Faced by QA Teams
Moreover, it’s not uncommon for a competent QA team to encounter many such errors as well, even though they may not have the same product audience appreciation.
Impact of Last-Minute Findings
In either case, the problem has already been “baked into the cake” and generally takes far longer to correct.
Temporary Solutions Due to Time Constraints
Worse yet, the solution is often a band-aid fix to meet deadlines, rather than taking the time to address the issue properly.
Post-Release Product Challenges
After a product is released, tech support analysts often encounter user issues that defy logical explanation. These problems go beyond simple user error, with many stemming from underlying data issues.
Common Response: Future Version Fixes
A frequent answer to these issues is, “It will be fixed in the next version.” Often, these are design flaws that cannot be acknowledged openly, leaving users with problems that require waiting for updates.
Resolution Process and User Impact
Correcting these issues typically involves exporting and importing data or running a one-time data cleansing utility. Either way, users are forced into a complex update process, resulting in a frustrating experience for end-users.
Handling Customer Requests for Data Models
The Challenge with Sharing Data Models
- When a sales rep identifies a sophisticated customer, they often request a data model or a roadmap of the database the product uses.
- As a product manager, I advocate for having this ready; however, many companies, or even VPs of product management, are reluctant to disclose it, claiming it protects the product’s intellectual property (e.g., the business glossary or the database design).
- I argue this is unnecessary since a potential customer with the knowledge to request a data model can easily reverse engineer it during the trial period. Failing to provide it may imply we’re hiding something, which has led to lost deals.
The Solution: Integrating Data Storage Design with Agile Methodology
- Early Data Requirements: Begin the agile process with scrum iterations that focus on collecting a basic understanding of data ontology needs and the best ways to persist that data.
- Data Model Creation: Develop a logical data modeling or diagram as an integral part of the application.
- Internal Review: Share the data model with product management for feedback to ensure alignment with overall goals.
- Quality Assurance Collaboration: Provide the data model to QA to support comprehensive testing across possible scenarios.
- Customer Transparency: Offer the data model to any potential customer who requests it to demonstrate the quality and transparency of the product.
The Importance of IDERA’s ER/Studio in Software Development
“For all such needs, IDERA’s ER/Studio ranks among the very best. It really should be a must-have tool for anyone building internal applications for their own company and for any company creating software to sell to others. If every application developer and software vendor integrated a tool like IDERA’s ER/Studio into their agile development process, the quality of all software everywhere would increase exponentially.”
Key Benefits of ER/Studio:
- Enhances internal application development
- Essential for software vendors
- Improves software quality across the board
