Page 1 of 2 12 Last
  • Jump to page:
    #1
  1. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69

    Testing; the key to heaven or hell?


    Hey all,

    This isnít specifically about one programming language, or programming languages in general now that I think about it, itís about testing your code. I didnít want to stick this in the lounge because it makes less sense to put it there than it does here IMO .


    This past week or so Iíve been thinking [more or less] exclusively about testing, what it means to be able to automatically test code and what the implications of this are.

    I have to admit itís a subject that Iím quite new to, so, if you would forgive me my ignorance here. If you have any comments or questions just shout out; even more so if you think Iím talking crap.


    So is automated testing really the answer, and why is testing (in one form or another) being pushed so much lately?

    Testing of course has always been emphasized (not just in software development), but this last year or so it seems to have shot up all over the place and blossomed. To be honest Iím not so sure I like the look or smell of these flowers!


    The first thing that hits me is the obligatory ďCOOLĒ. At first it seems more than useful to have one piece of code test another piece; think about all the time I could save myself; not to mention the happy feeling of closure .

    Ö But, and there always is one. This is only if the code doing the test can be guaranteed correct, which it canít. Now I can almost see people writing tests for tests if we continue down this road .


    I canít help thinking that maybe I just donít get it and Iím missing out. Thereís a lot to consider, itís a large subject and some of it is scary and or contradictory.

    Can you imagine a world where code that is otherwise correct gets flagged as having errors because of a fault within a test case? The programmer may well be fooled into introducing bugs just to pass!

    Even worse, what if some programmer, convinced that the code is right and the test wrong, adjusts some of the test cases?

    Well then weíve elevated a possibly simple problem to an uber bug Ė a bug that youíll almost certainly never find in a large system because everything is telling you that the problem is elsewhere. After all, weíve proven this code works fine havenít we.

    This sounds almost perverse donít you think.

    Whether or not this actually happens I canít say, but I suspect that it has and does quite often. Given enough time I think weíll start to hear about it but I wouldnít hold your breath for now.


    Interestingly many of the problems with common practices pointed out by advocates of test-driven development, are shared by tests.


    Itís been said by many great programmers that comments are largely a bad idea for a few different reasons; here are two of the usual reasons that I agree with .

    Comments duplicate knowledge that should already be present in the programs source.

    Comments are destined to become outdated, and so cause more harm than good in the long run.


    Well both of these reasons can certainly be applied to testing, maybe even more so.

    For any test to be truly worth itís weight in gold*, the level of detail has to be far, far, far greater than with comments, and this is detailed knowledge which must already exist in working code.

    Given that the second also seems to be true. If good tests need to go into such detail, then updating them accurately is a major job.

    If simple comments become outdated so easily then this implies that tests will certainly become stale at some point. That is of course if people didnít insist on using these tests to drive development.

    This brings me to my next point .


    We as humans (and presumably everyone reading this is) can only think about things in finite detail. Itís easy to see how and why being able to hack on some code and have the computer tell you when youíre done is desirable Ė this is no substitute for reasoning .

    It doesnít work because these tests can only tell you if youíre right [or wrong] for the cases you give them.

    If you want to write code that works you need to be able to reason about it, this is the only way that you can truly say a piece of code is correct; as in mathematics you need to prove it formally.

    We have to acknowledge that if tests arenít detailed enough, or are just plain wrong then the potential for evil is far greater than anything weíve had previously. If we donít trust our tests then theyíre useless to us, but we canít trust our code to be correct in general.

    How can we then double the code base and reasonably trust the extra code to tell us when weíre wrong?

    I donít think we can but maybe itís just me,

    Thanks for reading.

    Mark.


    * that is if tests weighed anything .
    programming language development: www.netytan.com Ė Hula

  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Jul 2006
    Posts
    2,270
    Rep Power
    1737
    By testing, I take it you mean unit testing. Obviously, you test your entire program in the end, but the Agile and XP, etc. methodologies brought unit testing to quite a spotlight. Approaching any hyped technology with skepticism is pragmatic. You do well to question it.

    Do you remember when XML a few years ago was expected to solve all the computing problems, along with world poverty and the like? Like XML, while unit testing itself is a useful tool, the hype can be misleading.

    It's best if you think of your code as black boxes being wired together. Unit tests are used to guarantee these black boxes function as advertised/designed/expected. They say nothing about performance, and they don't have a sanity check on your overall program design and logic. But especially in large projects, it helps to know when parts of your code severely malfunction.

    Remember, unit tests are best designed to check black box functionality. That means you should avoid writing ultra specific implementation tests. Limit your code coverage to only what is necessary. That is why you don't need to bear the onus of testing every bit of your code. It is actually detrimental as you note.

    You also realise that unit testing depends on the quality of the tests. By limiting yourself to black box functionality, you can avoid tripping over logic errors. And of course, the programmer should be wise enough to do so. As you note, poor unit tests can lead to even worse code.

    I'll give you an example. Let's say I write a linked list class in C++. I'll check the overall functionality. Can i add objects, and retrieve them? What about trying to retrieve from an empty list? I don't care about implementation in my unit tests. Sure, I don't test everything. I don't have a perfect guarantee my code works. But that's not what unit testing is for. It's a sanity check that my code behaves to a degree like expected.

    Comments on this post

    • netytan agrees : Thanks very much Oler1s. I didn't see this post when I looked earlier, strange!
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Feb 2004
    Location
    London, England
    Posts
    1,585
    Rep Power
    1373
    I do TDD (Test Driven Development) whenever possible, and the problems you cite are never an issue in practice.

    Automated tests do not prove that a program works - that is not what they are for, and proponents of automated testing do not claim that. What they do do is alert you when the program breaks in certain ways. If a test fails then you know that either the code is broken or the test is wrong. You then use the same debugging skills that you would use anyway to figure out which it is, then fix one or the other. As you say, tests are not a substitute for reasoning.

    If you write each test BEFORE you write the code it is testing then you get additional benefits on top of knowing when the code doesn't work. When you write a test for a piece of code that does not yet exist, you are focused on several questions that you would not be otherwise:
    * what does this piece of code need to do?
    * what data does it need to do that?
    * What is the result of this code - how will I know when its done it?

    These are all DESIGN questions - at this point you are not concerned about the implementation issues at all. TDD is a design methodology as much as a testing one. By writing the tests first you force each piece of code to be well structured so that it can be tested in isolation.

    Also with TDD you get IMMEDIATE feedback when your code breaks - seconds or minutes instead of waiting days, weeks or months for someone else to find the bug (perhaps a customer after it has been deployed). The gains from this are immense.

    Even if you do TDD you still want to do end-to-end testing since TDD will not catch bugs caused by the interaction between objects. You can do this with another automated tool such as Fit. You also want to have testers doing manual testing, since there are always things that cannot be tested automatically, such as usability.

    If you want to improve your code and testing then I recommend reading Test Drive Development By Example by Kent Beck.

    Dave
  6. #4
  7. fork while true;
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    May 2005
    Location
    England, UK
    Posts
    5,538
    Rep Power
    1051
    I'm a strong advocate of white box testing. Black box testing suffers from so many faults.

    When you're writing lisp particularly, you spend a long time ensuring the code is lofically correct. You're thinking of everything as data, you're logically working with data when you're writing the code. Your testing stage becomes mixed with your coding stage.

    Yeah, you gotta do a few testcases to make sure it still works, but a combination of the two is perfect.
  8. #5
  9. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Hey Dave,

    Firstly, thanks very much for the excellent reply. This was just the kind of thing I was after .


    I'm still learning about TDD Ė itís in my nature to take a skeptical view to anything that seemingly contradicts good programming practices, even if it does purport to offer a lot.

    In some ways it's seems like a good thing, in others it seems very bad. I've been writing simple tests for parts of my programs for a while now; one thing that Iíve noticed is that there is a lot of duplicated code, and no logical way to split it up and reuse it.

    This may be a bad idea I donít know. The way I write all my tests currently is as a logical description of the code being tested. Consequently I do feel that Iím duplicating a lot of information. Here Iím describing the purpose of the code logically; there Iím describing the purpose of the code functionally/declaratively (or however, depending on the language Iím using).


    A simple example, I have a higher-order function called make-between? that produces specialized versions of a predicate for testing if a value is within the given boundaries. As input it takes two predicates, which it uses to work this out.

    Code:
    > (define between? (make-between? > <))
    > (between? 3 1 5)
    #t  ; Truth value.
    > (between? 5 1 5)
    #f ; False value.
    >
    > (define inclusive-between? (make-between? >= <=))
    > (inclusive-between? 5 1 5)
    #t 
    >
    Iím sure you can imagine that when I write out the tests for these functions the tests are nearly identical. I can of course break these up to reduce the replication, but the break is unnatural.

    Iíve been play designing a macro for this that will let me to write a base test, and extend or update that with new test cases. It does seem to be more trouble that it should be right now though.

    Maybe Iíll change my mind?

    Iíll check out the book you pointed to when the workload lets up,

    Thanks again, you too LP,

    Mark.
    Last edited by netytan; October 2nd, 2006 at 04:38 PM.
    programming language development: www.netytan.com Ė Hula

  10. #6
  11. (retired)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 2003
    Location
    The Laboratory
    Posts
    10,101
    Rep Power
    0
    Hi guys, interesting thread so far.

    Originally Posted by Mark
    In some ways it's seems like a good thing, in others it seems very bad. I've been writing simple tests for parts of my programs for a while now; one thing that Iíve noticed is that there is a lot of duplicated code, and no logical way to split it up and reuse it.
    I've been using Django (excellent web app framework for python) a lot recently, and one of the things they do with their tests is to use them as a starting point for documentation and examples they can show to users. For example: here's the many-to-many database API.

    This is great - it's not repeating yourself so much as killing two birds (documentation/examples and testing) with one stone.

    I also think one of the better uses of tests is regression testing of bug fixes. I'm sure we've all worked on/seen apps which have those stupid little bugs that keep popping up and being reintroduced. If there was a reg. test to make sure that bug was not reintroduced, then you can immediately see it and squash it.

    --Simon

    Comments on this post

    • LinuxPenguin agrees : About docs, really good point, hadn't thought of that
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Feb 2004
    Location
    London, England
    Posts
    1,585
    Rep Power
    1373
    Here is how I handle that situation - other TDD programmers may do it differently.

    If I add a test that is a variation on a previous test, I usually start by cutting and pasting a copy of the entire test, rename it, and change whatever is needed in the body for the new test. Once it passes I then go back and refactor all the common code out into a helper function, so that both tests then call the helper and passing it different test data.

    If I then find I am writing lots of tests that all call the same helper function, then I refactor again and put the test data into an array and replace the tests with a single test that loops through the array and calls the helper function with different sets of data. this makes it very easy to add new test cases, but has the disadvantage that if one test case fails the others are not run. It also means that you cannot run individual tests (JUnit and the Python unittest module lets you run a single test by name - I dont know if the scheme unit test framework has that feature). Whether or not this is a problem depends on what you are testing - if the tests are fast enough and you always ensure that all tests pass 100% of the time then it is not a problem.

    There is a wealth of resources online about TDD, including articles with example programming sessions. For starters the are a bunch on the ObjectMentor site.

    Dave

    Comments on this post

    • woodyz agrees : The focus on TDD as opposed to merely Unit Testing is important
  14. #8
  15. Meatball Surgeon
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Jul 2004
    Location
    Elbow deep in code
    Posts
    2,056
    Rep Power
    1321
    Just my 2 cents, but i like testing the old fashion way; letting loose a group of fanatical users upon my code like a bunch of parahna.

    As much as this is frowned on now (to costly to have experienced users wasting thier time testing), knowledgable users will know how to test an app in its entirety.

    One of the worst things to run up against is to find out all your changes work (when tested individually be someone newbie) but when brought in into the scope of the tool and the users processes, that they become worthless.

    Comments on this post

    • LinuxPenguin agrees : I'd forgotten to mention that stage, but usually I don't bother with that until my own testing is complete and the app passes them all and vaguely works as it is supposed to
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2002
    Posts
    596
    Rep Power
    24
    Knowing what to test is an art that needs thought and it's very easy to write a useless test, to miss out on testing something or to test with the wrong data. QuickCheck is an interesting haskell testing library that removes the programmer to a level above - you provide a specification for a function, not the tests. The tests are generated and run for you with a range of random data including corner cases, fenceposts etc (the data can be controlled). Because haskell is a purely functional language with no side effects (apart from impure code) this effectively verifies the function - if your code does include side effects you're back to painstakingly crafting your own unit tests, and it's at that point that you realise how difficult it is to write a test which is really effective under all conditions unless you avoid side effects.
  18. #10
  19. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Nice posts guys,

    Jamie: I think this will be one area where functional programming languages just come out on top in the long run. If there is one thing that could push them into mainstream it's probably this.

    It's a bit of a shame that functional programming isn't very popular since functional languages tend to be far more beautiful than there imperative oppressors, in my honest opinion.

    That said functional programming is becoming more and more common. That's always a plus .

    Mark.
    programming language development: www.netytan.com Ė Hula

  20. #11
  21. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2006
    Location
    San Diego, USA
    Posts
    527
    Rep Power
    149
    Originally Posted by netytan
    .. This past week or so I've been thinking [more or less] exclusively about testing ... if you would forgive me my ignorance here.. talking crap...
    I, for one, forgive you, son.
    Originally Posted by netytan
    So is automated testing really the answer, and why is testing (in one form or another) being pushed so much lately?
    The answer to what? What is the question? There is only one alternative to automatic testing, and that is manual testing (unless you consider no testing an acceptable alternative). I vote for automatic testing every day whenever and wherever possible. That is one reason why I program computers: To do things automatically so I don't have to do them myself.
    Testing is being "pushed" a lot lately, just like "brushing your teeth after every meal" or "check your blind spot before making a lane change" is "being pushed". These are all good practices to follow.
    Originally Posted by netytan
    Testing of course has always been emphasized (not just in software development), but this last year or so it seems to have shot up all over the place and blossomed. To be honest I'm not so sure I like the look or smell of these flowers!
    Lets change that to "this last 10 years or so" and I'll go along with it. If you are specifically referring to automated unit testing using tools such as
    SUnit, JUnit, NUnit, xUnit then perhaps the "last 8 years or so" fits. The idea of self-testing code has also been around for a LONG time. And I, personally LOVE THE SMELL.
    Originally Posted by netytan
    The first thing that hits me is the obligatory "COOL". At first it seems more than useful to have one piece of code test another piece; think about all the time I could save myself; not to mention the happy feeling of closure .
    This is exactly the point. And I think this is where you will benefit most when you get an understanding of what Unit testing is all about.
    Originally Posted by netytan
    ... But, and there always is one. This is only if the code doing the test can be guaranteed correct, which it can't. Now I can almost see people writing tests for tests if we continue down this road .
    What kind of guarantee are you looking for??? The basic concept of unit testing is that each time a change, a "bug", or unwanted result is detected, a new test is added, and the code is fixed to correct the bug. The whole purpose of this is that you will slowly be building up a library of specific tests for each unit that you kick-off each time you make changes - all those previous tests are there to alert you if you have broken anything that you already have a test for. And as practical matter, unit testing tests a very finite block of code, and I have found I have no trouble proving that a block of code works exactly as intended via unit tests. If it seems to you that this is impossible, then I strongly suggest you work with it for a while and practice the techniques available in tools like JUnit or NUnit. I am certain you will gain a confidence in the results that is much more satisfying than you are currently able to muster since it appears from what you are saying that are not using unit testing.
    Originally Posted by netytan
    I can't help thinking that maybe I just don't get it and I'm missing out. There's a lot to consider, it's a large subject and some of it is scary and or contradictory.
    I hope you aren't really finding it to be scary. But I, too, can't help thinking that you are missing out. I have been using automatic progressive unit tests for about 10 years, and can't imagine programming without them. My thinking on this topic was clarified by reading Martin Fowlers refactoring book and Kent Becks various materials on the topic about 6 or 7 years ago. Look up these guys, and any other related reading and start practicing the use of automated unit testing - you will never go back.
    Originally Posted by netytan
    Can you imagine a world where code that is otherwise correct gets flagged as having errors because of a fault within a test case? The programmer may well be fooled into introducing bugs just to pass!
    I don't see this at all. I can imagine that there are developers who write code just to get past a faulty test, but I haven't ever worked with any that I know of. But I HAVE seen a lot of fixes that fixed the problem at hand but broke other things that weren't discovered during testing because there was no comprehensive automated unit or regression testing. This is precisely what we are attempting to address with these practices. Like they say: Don't throw out the baby with the bathwater.
    Originally Posted by netytan
    Even worse, what if some programmer, convinced that the code is right and the test wrong, adjusts some of the test cases?

    Well then we've elevated a possibly simple problem to an uber bug - a bug that you'll almost certainly never find in a large system because everything is telling you that the problem is elsewhere. After all, we've proven this code works fine haven't we.
    Again... what world do you live in???
    Originally Posted by netytan
    This sounds almost perverse don't you think.
    Your analysis is based on a faulty understanding of the purpose and use of these various testing practices.
    Originally Posted by netytan
    Whether or not this actually happens I can't say, but I suspect that it has and does quite often. Given enough time I think we'll start to hear about it but I wouldn't hold your breath for now.

    Interestingly many of the problems with common practices pointed out by advocates of test-driven development, are shared by tests.

    It's been said by many great programmers that comments are largely a bad idea for a few different reasons; here are two of the usual reasons that I agree with .

    Comments duplicate knowledge that should already be present in the programs source.

    Comments are destined to become outdated, and so cause more harm than good in the long run.

    Well both of these reasons can certainly be applied to testing, maybe even more so.
    What? You are saying that testing duplicates knowledge and becomes outdated and does more harm then good in the long run. Either you are joking or have never worked on a project with a testing process in place. I don't get the "duplicate knowledge" thing at all, and fill me in: How do tests become out of date??? Tests are a dynamic thing - if some functionality has been changed, removed, or added on the project then the tests are changed to reflect the changes. It will be discovered immediately when the tests fail that they are out of sync with the current state of the project.
    Originally Posted by netytan
    For any test to be truly worth it's weight in gold*, the level of detail has to be far, far, far greater than with comments, and this is detailed knowledge which must already exist in working code..
    What knowledge are you talking about? The code knows nothing. The code can communicate its intent to a developer who has access to the source code via good naming and coding practices... but other than that, the code never knows if it is DOING THE RIGHT THING - it only knows how it goes about TRYING TO DO WHATEVER IT IS SUPPOSED TO BE DOING.
    Originally Posted by netytan
    Given that the second also seems to be true. If good tests need to go into such detail, then updating them accurately is a major job..
    Again... what world do you live in???
    Originally Posted by netytan
    If simple comments become outdated so easily then this implies that tests will certainly become stale at some point. That is of course if people didn't insist on using these tests to drive development..
    For your own sake I hope you get a better understanding on this stuff... how in the world do you come to the conclusion that tests will become stale??? That is like saying that the code itself will become stale. Both the code and the tests are living things, so to speak. If tests are ignored and never actually used, then of course they might become "stale", but what would that matter since they aren't being used anyway? And further, it would be likely that the enterprise depending on the code would be out of business soon enough if they haven't worked out decent testing at all levels of the development process.
    Originally Posted by netytan
    This brings me to my next point .

    We as humans (and presumably everyone reading this is) can only think about things in finite detail. It's easy to see how and why being able to hack on some code and have the computer tell you when you're done is desirable - this is no substitute for reasoning .

    It doesn't work because these tests can only tell you if you're right [or wrong] for the cases you give them.

    If you want to write code that works you need to be able to reason about it, this is the only way that you can truly say a piece of code is correct; as in mathematics you need to prove it formally.

    We have to acknowledge that if tests aren't detailed enough, or are just plain wrong then the potential for evil is far greater than anything we've had previously. If we don't trust our tests then they're useless to us, but we can't trust our code to be correct in general.

    How can we then double the code base and reasonably trust the extra code to tell us when we're wrong?

    I don't think we can but maybe it's just me,
    .
    Wow. With this sort of circular reasoning, you should start a relegion or a government.

    My suggestion: Find someone nearby who can show you how this stuff works and how they are actually used to improve code correctness, productivity, and sanity. I use TDD as a thinking tool, and a lot of developers I have worked with do the same. If used this way, I have found I can move through complex problems quickly and with good results. In other words, it works as a design tool. The typical projects I work on include dozens of developers, usually on teams all over the world. Without decent testing processes, it would be impossible.

    Anyway... have fun.

    Comments on this post

    • netytan disagrees : Read the original post again, maybe you'll notice the reason it was written like that.
    Last edited by woodyz; October 26th, 2006 at 04:13 PM.
  22. #12
  23. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Wow, someone has a very disrespectful tone, and seems to have missed the point of my original post entirely: to stimulate arguments for testing .


    Why would I ever want to do that? Could it have to do with testing the waters?

    I wanted to get at peoples real opinions on the subject, and I got some very nice posts in return. Not all were for or against TDD. This isnít such a clear-cut subject; if you really think it is then sadly youíre mistaken.


    You really canít infer what is good or bad practice from what everyone else is doing!

    Iíd personally class the decision to question the effectiveness of testing as a good one, because lets face it, theyíre not all that effective*. As you yourself said, and as we all knew: they only catch the errors that you we about or have otherwise accommodated for.

    If youíve done any formal training then you should remember learning about proving theorems, usually in some discrete mathematics or algorithms course. Generations of study in mathematics have shown that this is the reliable way to prove if things like this work as intended!

    Testing against a limited number of cases just doesnít square up to that, but it is easier.

    For the most part Iíd rather spend the extra time to produce a semi-formal proof, than write up (and maintain) dozens of tests. The results tend to be far better in my opinion because I inevitably get some new insight into the problem domain by doing this.


    If I just followed the crowd Iíd probably end up working with crap like insert industry standard language here, on some dime a dozen business application. The opportunity is certainly always there no .


    I canít say that I commonly work with dozens of people, but any well-written code should be resistant to this kind of work. If you make good use of abstraction and high cohesion, then no matter how much you change how your code works you canít break anyone elseís work Ė Think encapsulation on a massive scale .

    If the code is badly crafted then sure, make sure that you run your tests after every change!


    On encoding knowledge: thatís what writing code is all about, we are (or should be) encoding all the knowledge and meta-data that we have about the project. You may be worried for me, but Iíd be even more worried that you donít know basic CS concepts like .

    Note: you may like to follow up reading about this topic with research into declarative programming methodologies.


    Ultimately. I donít have to justify the decisions I make to forward my own learning to you. Anyone who knows me can tell you that Iím a very capable, well-informed programmer .

    Take care,

    Mark.


    *I can think of one very good example of where not enough people questioned a solution: OOP, especially statically typed OOP.

    Comments on this post

    • woodyz disagrees : Read my original reply to your original post again, maybe you'll notice the reason it was written like that.
    programming language development: www.netytan.com Ė Hula

  24. #13
  25. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2006
    Location
    San Diego, USA
    Posts
    527
    Rep Power
    149
    Originally Posted by netytan
    I have to admit itís a subject that Iím quite new to, so, if you would forgive me my ignorance here. If you have any comments or questions just shout out; even more so if you think Iím talking crap.
    It appears you weren't really sincere about this line!
    Originally Posted by netytan
    Wow, someone has a very disrespectful tone, and seems to have missed the point of my original post entirely: to stimulate arguments for testing .
    Sorry if I offended you, but you had asked for comments, and I didn't mean any disrespect. But I certainly took what you stated above to mean that you are "NEW TO" the subject of testing, and ignorant about it (your words).
    Originally Posted by netytan
    Why would I ever want to do that? Could it have to do with testing the waters?
    I wanted to get at peoples real opinions on the subject, and I got some very nice posts in return. Not all were for or against TDD. This isnít such a clear-cut subject; if you really think it is then sadly youíre mistaken.
    I never said it was clear cut. I said that you might find it to be very useful if you learn how to properly use it. You seem to be more interested in bashing it without any practical experience with it.
    Originally Posted by netytan
    You really canít infer what is good or bad practice from what everyone else is doing!
    That's right. I don't infer what is good or bad. If something looks useful, I give it a try - if I like it I use it. It makes little sense for someone who is new to it to be bashing it without trying it.
    Originally Posted by netytan
    Iíd personally class the decision to question the effectiveness of testing as a good one, because lets face it, theyíre not all that effective*. As you yourself said, and as we all knew: they only catch the errors that you we about or have otherwise accommodated for.
    What "are not all that effective"? All tests? All practices? All testing methodologies? All questions? What?
    Originally Posted by netytan
    If youíve done any formal training then you should remember learning about proving theorems, usually in some discrete mathematics or algorithms course. Generations of study in mathematics have shown that this is the reliable way to prove if things like this work as intended!
    Of course, writing the right code is critical. But proving that you have writen the right code is also critical.
    Originally Posted by netytan
    Testing against a limited number of cases just doesnít square up to that, but it is easier.
    Now who is using a "very disrespectful tone" and missing the whole point?
    Originally Posted by netytan
    For the most part Iíd rather spend the extra time to produce a semi-formal proof, than write up (and maintain) dozens of tests. The results tend to be far better in my opinion because I inevitably get some new insight into the problem domain by doing this.
    Of course, it almost goes without saying that you need to work the way you find is most productive for you. But your comments indicate that you do not understand the motivations of TDD and unit testing, and I am just suggesting it might be worth your while to learn about it and try it out for a while. It has serverd me well, and it might be useful for you.
    Originally Posted by netytan
    If I just followed the crowd Iíd probably end up working with crap like insert industry standard language here, on some dime a dozen business application. The opportunity is certainly always there no .
    Yes, sure... if all your friends jumped off a cliff would you jump too? Of course not. An I am not suggesting you just "follow the crowd". YOU were stating that you are just starting to look into this stuff, and I was suggesting you might find it useful. Touchy fellow!
    Originally Posted by netytan
    I canít say that I commonly work with dozens of people, but any well-written code should be resistant to this kind of work. If you make good use of abstraction and high cohesion, then no matter how much you change how your code works you canít break anyone elseís work Ė Think encapsulation on a massive scale .
    I never said anything about breaking someone elses work.
    Originally Posted by netytan
    If the code is badly crafted then sure, make sure that you run your tests after every change!
    Again, I took you at your word that "I have to admit itís a subject that Iím quite new to, so, if you would forgive me my ignorance here". Here you seem to be saying that running tests after every change is only useful on badly crafted code. This is obviously your view of the process. You could have started out with that so at least we would know your motivation.
    Originally Posted by netytan
    On encoding knowledge: thatís what writing code is all about, we are (or should be) encoding all the knowledge and meta-data that we have about the project. You may be worried for me, but Iíd be even more worried that you donít know basic CS concepts like ..
    Ah. Like what basic CS concepts? I have been developing software for a good long time - about 25 years or so, and it always amazes me at how much I don't know, and how much I still have to learn. One thing I do know is there are at least 1000 right ways to do anything, and whatever I might think works, someone else will say it doesn't. TDD is something I use daily, and it works well for me.
    Originally Posted by netytan
    Note: you may like to follow up reading about this topic with research into declarative programming methodologies.
    Thanks for the tip. Which ones do you recommend? I was starting to guess that your motivation in this post had nothing to do with the focus the industry has on testing.
    Originally Posted by netytan
    Ultimately. I donít have to justify the decisions I make to forward my own learning to you. Anyone who knows me can tell you that Iím a very capable, well-informed programmer ..
    I never asked you to justify anything to me. And I certainly don't see any need to ask people who know you about your abilities - I assume that you are capable. In your first post you seemed to be asking about something you were just looking in to and din't know much about. Now you say "I wanted to get at peoples real opinions on the subject". Well... didn't I give you my real opinions? Sorry I wasted your time. If you ever actually want to see how TDD works or can make the developement process work better - perhaps we can cover that. If you just want to bash it - that's your right. And you further state that I missed "the point of my original post entirely: to stimulate arguments for testing". Well, I think I have stimulated something, anyway.
    Originally Posted by netytan
    *I can think of one very good example of where not enough people questioned a solution: OOP, especially statically typed OOP.
    Now this is my favorite little jab out of the whole bunch! Very cute. Play the "OOP Sucks" card. That ought to rile up some folks. Well done!

    Apparently you take offense at my "tone", and I am sorry for that, but you can ask anyone who knows me: I am a very annoying, ill-mannered lout who always sounds like he's full of it - and most of the time they are right.

    Have fun.
    Last edited by woodyz; October 26th, 2006 at 04:50 PM.
  26. #14
  27. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Yes I am/was pretty new to TTD and I did ask for any comments. You must agree that thereís a difference between constructive and ďAgain... what world do you live in????Ē among others.

    Thatís not useful to me, or fun, just someone being uppity out of the blue. I have no doubt that you know your stuff but please drop the ďyou are my subordinateĒ attitude. The content wasnít bad at all .


    When I created this thread I wrote it this way to get some passion on the subject. Iíve found that it gets people more involved in the topic: better responses, more passion and less corporate babble.

    Then of course I enjoy taking the opposing side of a discussion like this .

    Originally Posted by woodyz
    ask anyone who knows me: I am a very annoying, ill-mannered lout who always sounds like he's full of it - and most of the time they are right.
    LMAO, believe me, Iím sure people could say worse things about me. Particularly Linux Penguin . No worries man we're good.


    On encoding knowledge: this isnít a new thing, I thought everyone knew it. On that I guess I was wrong. This idea is very simple; Iíd bet youíve been doing it implicitly for years (or Iíd hope so).

    When you write any code youíre building up knowledge, about the domain, the constraints, any insight you had. Itís all there under the surface.

    As with any duplication the same problems apply. The only difference here is that the duplication in this case is much subtler.

    When you write a test you are effectively describing the action of the code that youíre writing or have written. When you change one you now have to change the other as you noted.

    Youíve effectively described the same function twice, in two discrete/disparate forms.

    I doubt anyone would disagree that duplication is bad (that much is considered common knowledge now) be it duplication of code, or duplication of knowledge as above.


    On the OOP comment: Let me preface this by saying that I donít think OOP sucks, exactly.

    The OO view is a nice way to look at the worldÖ if you want to do simulations itís natural; itís the right tool for the job. We use it for pretty much everything!

    From a conceptual point of view you must see that OOP doesnít even address the problems itís supposed to fix, all of the familiar problems with imperative/procedural programming are still present, weíve just put them into boxes with pretty labels.

    Instead of these problems being present in a single program they get spread out and cut up. Theyíre still there.

    You can see classes as being nothing more than separate programs unto themselves.

    Next note that OOP doesnít make writing reusable code easy, you have to put a lot of work into your class model to get the most out of objects.


    Iím not pretending to be unbiased in either point, but I have put a lot of thought into these subjects.


    I think thatís everything .

    Enjoy,

    Mark.
    programming language development: www.netytan.com Ė Hula

  28. #15
  29. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2006
    Location
    San Diego, USA
    Posts
    527
    Rep Power
    149
    Originally Posted by netytan
    Yes I am/was pretty new to TTD and I did ask for any comments. You must agree that there's a difference between constructive and "Again... what world do you live in????" among others.
    Give me a break! You dish it out worse than me in half your posts on this site - I was just using an expression.
    Originally Posted by netytan
    That's not useful to me, or fun, just someone being uppity out of the blue.
    Perhaps, but by that time in my response I was starting to wonder if you have had any real-life programming job experience, or merely student and hobby level stuff. That was in response to your comment about the amount of work it takes to maintain the tests: Which is virtually zero. In practice, the tests generally build up over time, and existing tests need almost no maintenance..
    Originally Posted by netytan
    I have no doubt that you know your stuff but please drop the "you are my subordinate" attitude. The content wasn't bad at all .
    You are reading attitude into my comments. Just read every other word, and you will get the true meaning.
    Originally Posted by netytan
    When I created this thread I wrote it this way to get some passion on the subject. I've found that it gets people more involved in the topic: better responses, more passion and less corporate babble.
    So then whats the big deal? Do you only want people to respond who happen to know that you are just trying to get passionate responses, but if they are even slightly interlaced with "attitude" you get offended?
    Originally Posted by netytan
    Then of course I enjoy taking the opposing side of a discussion like this .
    This is always a lot of fun, of course... why else would we do it?
    Originally Posted by netytan
    LMAO, believe me, I'm sure people could say worse things about me. Particularly Linux Penguin . No worries man we're good.
    Now on to the real stuff
    Originally Posted by netytan
    On encoding knowledge: this isn't a new thing, I thought everyone knew it. On that I guess I was wrong. This idea is very simple; I'd bet you've been doing it implicitly for years (or I'd hope so).
    You misunderstood, probably because I was unclear: I was questioning the type of knowledge you seem to think the code itself holds, and your clarification below indicates to me that I was not mistaken.
    Originally Posted by netytan
    When you write any code you're building up knowledge, about the domain, the constraints, any insight you had. It's all there under the surface.
    This knowledge is as I stated earlier: The code has no way of expressing that it is doing the right thing, it can only express what it it trying to do, whether it is the right thing or not.
    Originally Posted by netytan
    As with any duplication the same problems apply. The only difference here is that the duplication in this case is much subtler.
    When you write a test you are effectively describing the action of the code that you're writing or have written. When you change one you now have to change the other as you noted.
    Perhaps we have to leave it here - I disagree with you on this. By this way of thinking, the user of the code (the end user of the app) is also a duplication of knowledge, since he has knowledge of the action of the code!! So no users allowed! I am a bit puzzled on how you can come to the conclusion you are making. But I am easily puzzled. That is why I am just a lowly software engineer
    Originally Posted by netytan
    You've effectively described the same function twice, in two discrete/disparate forms.
    As I see it, with unit tests I am defining how I want the code to respond, and in the code I am providing a the mechanism that provides the response, so to speak. Equating this to the duplication that comes from code comments is not a fair comparison. Well written and named methods eliminate most of the reason for commenting since the intent of the code is clear to the reader of the code. But the RESULT of the code is not part of that knowledge - only the INTENDED RESULT is. By your reasoning, the only actual entity that can discover there is a problem will be the end user (but luckily a little earlier on I have eliminated him so he will no longer be a problem) - you seem to think that proper thinking and writing of the code cannot fail but to create a perfectly working solution. Of course, good thinking up front is essential, but it is only part of the process.
    Originally Posted by netytan
    I doubt anyone would disagree that duplication is bad (that much is considered common knowledge now) be it duplication of code, or duplication of knowledge as above.
    Again - what world do you live in. NO WAIT, I take that back... uh, I mean... I disagree that there is some inherent problem with duplication of knowledge in the use of Unit Testing. Again - we are just going to have to leave it there, since we are on two sides of the fence on that one.
    Originally Posted by netytan
    On the OOP comment: Let me preface this by saying that I don't think OOP sucks, exactly.
    The OO view is a nice way to look at the world... if you want to do simulations it's natural; it's the right tool for the job. We use it for pretty much everything!
    From a conceptual point of view you must see that OOP doesn't even address the problems it's supposed to fix, all of the familiar problems with imperative/procedural programming are still present, we've just put them into boxes with pretty labels.
    I can't use the "what world do you live in" thing, so...
    Originally Posted by netytan
    Instead of these problems being present in a single program they get spread out and cut up. They're still there.
    You can see classes as being nothing more than separate programs unto themselves.
    Next note that OOP doesn't make writing reusable code easy, you have to put a lot of work into your class model to get the most out of objects.
    Hmmm. Are you saying that a programmer must understand what he/she is doing to make OOP work? Yes. I agree on this one! HEY EVERYBODY - We agree on something!!!
    Originally Posted by netytan
    I'm not pretending to be unbiased in either point, but I have put a lot of thought into these subjects.
    Okay - I'll buy that. I can tell you have thought a lot about these things (the "duplicate knowledge" theory and hating OOP). That is clear. Here is what I think about OOA, OOD, and OOP: It works very well, it helps me solve a lot of complex problems in a clear, easily maintained, and highly extensible manner. Modern thinking on OOP, Ala Robert Martin, and all the various "gurus" and authors on languages such as Ruby, Java and C# have pretty much layed out a clean path for maximizing the value of OOP. That isn't to say that something better isn't out there.
    Originally Posted by netytan
    I think that's everything .
    Well. Just about, grasshopper (dang, sorry for the "subordinate" reference there ... that was just an uppity slip). My opinions and observations are generally valuable only to me. I like to cast them about here and there in the various forums just for fun. So here is a little story about how things work: A few years back I was working for a company that was helping to invent the hard drive (now called hard disk). I was hand making prototype hard drive heads. They were about the size of a US 5-cent piece. I cast them in resin from molds I made from machined metal - we needed a lot of these. We then wired them up, and attached them to our prototype hard drives, which were also hand made. There were numerous, very intelligent engineers and scientists working on this - I was just a kid, and these folks were giants, brain-wise. Some of them geniuses, and most of them came from the premier computer hardware and software companies in the world at that time. Top people (or Neanderthals as we were called back then). And they calculated, proved theorems, even had a little toss at discrete math once or twice just for the fun of it... but when it all came down to it, they invented the modern hard disk (and floppy disks too, you remember floppy disks?) based on endless testing of things working in the real world. We did wind tunnel tests (by blowing on the apparatus with our own breath), and knocked the side of the apparatus to get the heads to crash, and then they figured out what could be done to make it better the next time. That was pretty much the pattern - Design the best thing possible, make it, test it, destroy it in the testing, figure out what happened... or at least what we thought was happening, and then start all over - hundreds of times, thousands for some things. I learned a lot from that, and it has been useful to me. Even for software development where at least the tests don't destroy the product. However - it probably is of little value to you to hear about it. But at least I tried.
    Originally Posted by netytan
    Enjoy,
    Mark.
    You too.
    Last edited by woodyz; October 26th, 2006 at 08:48 PM.
Page 1 of 2 12 Last
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo