Pointing domains to IP's

This may seem like a no-brainer to some, but I've been struggling with it lately.

I have a server at home with a dynamic IP. I am familiar with using the various dynamic DNS services to point a domain name to an everchanging IP address.

Now, suppose I have a domain called mydomain.com and it has been registered. I have pointed that domain to my server and it works fine.

Now, what I want to be able to do is assign sub domains without relying on support from a registrar to do so. For instance, I'd like to setup my machines so that I can access them through the following domains:

machine1.mydomain.com
machine2.mydomain.com
machine3.mydomain.com
etc...

This is where I get stuck. I'm not sure what to use in order to do this. I am guessing I will need to setup the server to be a DNS server. From there I could use my server to assign various sub domains off of my existing domain?? Does that make sense??

If I do need to setup DNS, can I setup the main DNS server using a sub domain? In other words, do any machines on the network have to be labeled as mydomain.com without the sub domains? I want to main server to be setup as machine1.mydomain.com if possible.

Any help would be greatly appreciated!

You need to delegate your domain to, say ns1.mydomain.com. As far as whether you can get subdomains from your current DNS host you need to tell us your real domain.
Asking for DNS help using mydomain.com is unhelpful and wasting our time.

If your IP address changes several times a week then it is not worth your while to host a DNS server without dynamic delegation. However, there are services out there who will host your domain and offer dynamic IP services.