Page 2 of 2 First 12
  • Jump to page:
    #16
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5
    Wow! When i made a copy of abc.jar on the source server and then tried transferring it, it worked! I am really curious to know why this worked and the original did not?
  2. #17
  3. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    A strange corruption of the data in some way? Why it would 'clear up' with a local copy and not when being scp-ed ... not a clue. So far as I know the same file system drivers would be involved to read the file, but I could be wrong, or they may be the same drivers but be acting in a slightly different way.

    The good thing is that it is, for now at least, solved!
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  4. #18
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5
    I guess it's not totally solved from what i hear from the developer. Some strange behaviour is happening with other files that are related (may be created/modified on HP machine) to HP-UX machine. The build script that creates this file uses Make and related stuff, which i am not very comfortable working with. I need to do a more thorough investigation at least to identify some concrete pattern.

    But anyways, thanks a lot for your time Simon! I really appreciate it. Thanks kicken for your suggestion!
  6. #19
  7. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    Happy to try and help.
    Totally off-the-wall suggestion - no-one accesses the files at the source end with something like CIFS/Samba do they, or some other mechanism that might try and apply non-standard permissions to the files? By non-standard I also include ACLs.
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  8. #20
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5
    I am sorry but I missed mentioning that the source directory is an NFS share. And yes, people do access the share using Samba as well.

    I am not sure about the ACLs. 'getfacl' wasn't working on my Linux box, which is the source machine. I'll have to check.
  10. #21
  11. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    As the error appears to be permission-based, that may have a bearing on the issue.
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  12. #22
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5

    'Access Denied' issue while copying binary


    This issue was on the back burner for a while. All this while, I got to know the system better and what exactly is happening. I have changed the Title here to reflect the issue at hand.

    The issue is not just with SCP alone. First of all, there are few things that I would like to clear up:
    -------------------------------------------------
    1. We have a workspace (/wksp/project/) which is actually an NFS share. This share is mounted on different UNIX/Linux flavors (HP/Sun/AIX/Linux). Builds happen on all the given machines (platforms) but in this common mounted workspace. When binaries are built on this share, they are stored under different sub-directories as per platform names (HP/Sun/AIX/Linux).

    2. We have another share called /auto/allbuilds, which is again an NFS share. This share is also mounted on all the above platforms. This is a publishing location. This is where the binaries are copied once they get created in the first share (/wksp/project/) stated above. They get copied under a similar structure (platform-wise) under this publish share (/auto/allbuilds)

    For example,
    /wksp/project/release/hp-ux/ora.tpi
    gets published at
    /auto/allbuilds/20111029/Integration/hp-ux/ora.tpi

    /wksp/project/release/aix-ppc/ora.tpi
    gets published at
    /auto/allbuilds/20111029/Integration/aix-ppc/ora.tpi
    -------------------------------------------------

    Now moving to the issue:
    Users (QA/Dev) who have to use the built binaries do not have access to the first NFS share (/wksp/project/) where the binaries gets created. Only when we copy (publish) the binaries to the second share(/auto/allbuilds/...), can they access it.

    Now the problem is that although they can copy the AIX/Linux/Solaris binaries from respective sub-directory under /auto/allbuilds/…, they get ‘Acces Denied’ error when accessing the HPUX binary, which is kept under /auto/allbuilds/20111029/Integration/hp-ux/ora.tpi.

    Both the NFS share are exported from a Sun Solaris server.

    I will now show the permission bits present on both these NFS shares. I am executing the commands on Sun Solaris box (NFS server) so if there will be any ACL, it will show that with a + sign at the end of file permission.

    bash-3.00$ pwd
    /wksp/project/
    bash-3.00$
    bash-3.00$ find . -name ora.tpi -exec ls -ltr {} \;
    -r-xr-xr-x 1 esm2 other 73855426 Oct 29 08:40 ./aix-ppc64/ora.tpi
    -r-xr-xr-x 1 esm2 other 49484255 Oct 29 08:40 ./hpux-ia64/ora.tpi

    As you can see above, the location where the file gets created for each platform has read permission for all, which is fine. Now we will see the permission for the same binaries on the publish share.

    bash-3.00$ cd /auto/allbuilds/20111029
    bash-3.00$
    bash-3.00$ find Integration -name ora.tpi -exec ls -ltr {} \;
    -r-xr-xr-x+ 1 esm2 other 73855426 Oct 29 08:43 Integration/aix-ppc64/ora.tpi
    -r-x---r-x+ 1 esm2 other 49484255 Oct 29 08:43 Integration/hpux-ia64/ora.tpi

    As you can see above, there is issue with the permission on HPUX binary. Also, ACL is set as shown by the + sign.

    Some background information:
    The script that builds the binaries and does the publishing resides on a Linux box. It does SSH to different platforms and starts the build on each platform. As mentioned earlier, all platform use the common mounted workspace (/wksp/project) and a common mounted publish share (/auto/allbuilds).

    The copy operation does not use the –p option for preserving the permission bits. Yesterday, I thought of trying out the same copy operation with –p option set. I logged on to the HPUX box with the same user that the script uses while doing SSH, and then fired the copy command:

    cp –p /wksp/project/release/hp-ux/ora.tpi /auto/allbuilds/20111029/Integration/hpux-ia64

    …but I got the same result (as expected) that I get while running the script i.e.,

    -r-x---r-x+ 1 esm2 other 49484255 Oct 29 08:43 Integration/hpux-ia64/ora.tpi

    Though the source file has ‘r-x’ for group, when the files gets copied to the destination, those bits are missing for the group. Looks like there is some issue with ACL on the publishing share (/auto/allbuilds). Someone told me that if ACL is set, it will override the default permissions. Looks like the same is happening here but I am not able to figure out exactly where to look for.

    I checked the ACL on the NFS share:

    bash-3.00$ getfacl /auto/allbuilds/20111029/Integration/aix-ppc64
    # file: /auto/allbuilds/20111029/Integration/aix-ppc64/
    # owner: esm2
    # group: other
    user::rwx
    user:500:rwx #effective:rwx
    group::r-x #effective:r-x
    mask:rwx
    other:rwx
    default:user::rwx
    default:user:500:rwx
    default:group::r-x
    default:mask:rwx
    defaultther:r-x

    I get the same result as above for /auto/allbuilds/20111029/Integration/hpux-ia64

    Any help?


    Thanks,
    Gaurav
  14. #23
  15. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    Oooookay ... let's see if we can't try and unravel some of this ... but, I, for one, will need several cups of coffee and a couple of confirmations/explanations of the setup:

    1) The file systems that provide the /wksp/project and /auto/allbuilds are physically attached to a Solaris server that is acting (for these) as an NFS server.
    2) Those nfs mounts are mounted by a variety of other *nix-based systems
    3) All the 'admin' type work done with the contents (the builds, the copying, etc.) is done from a Linux server.

    Would those be accurate statements?

    Are these file systems/directories 'mounted' anywhere else, such as via Samba/CIFS?

    Due to the date part of the path name I am guessing that (the first build at least) creates the directory where the build is copied to.
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  16. #24
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5
    Thanks for your reply Simon!

    >>> 1) The file systems that provide the /wksp/project and /auto/allbuilds are physically attached to a Solaris server that is acting (for these) as an NFS server.

    Yes. Both the shares are directories physically present on Solairs server.


    >>> 2) Those nfs mounts are mounted by a variety of other *nix-based systems

    Correct!


    >>> 3) All the 'admin' type work done with the contents (the builds, the copying, etc.) is done from a Linux server.

    Yes, the build script is fired from the Linux server. This script that runs on this Linux box connects to all other machines (Linux/AIX/Solaris/HPUX) using SSH and fires command on those machines. The command fired on each platform creates the binaries and then also copies it to the publish location, which is again an NFS share mounted on all those given platforms.


    >>> Are these file systems/directories 'mounted' anywhere else, such as via Samba/CIFS?

    Yes, they are mounted using Samba/CIFS as well.


    >>> Due to the date part of the path name I am guessing that (the first build at least) creates the directory where the build is copied to.

    You're right! At the destination (where the binaries are copied), the directories are created at the time the script is run. It's on a daily basis.
  18. #25
  19. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    What a lovely mess! When you start mixing and matching nfs/CIFS across *nix platforms it can get really messy.
    Do you have matching usernames/uids across all platforms (natively via passwd or globally via ldap, etc.?). What are the ownerships and permissions of the directories involved - natively on the Solaris nfs server and also on the nfs clients?
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  20. #26
  21. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5
    Sorry for the delay in reply.

    I have few things to clarify. Please correct me if I am getting something wrong.

    I agree there are issue in both the *nix platforms and on Windows (where the publish location is mounted) but when we are only focusing on the issues on non-Windows platforms, how should Samba/CIFS affect that?

    As far as permissions are concerned, all platforms have a single NIS user and the same group except for one flavor (Linux). However, there are few issues there as well. I have asked the IT team to remove all unwanted entries from /etc/passwd and /etc/group besides any other so that we can be completely sure that all machines are using same, consistent user and its permissions. Once that’s done, which should happen in this week, I will then update you.
  22. #27
  23. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    NIS???? Wow, not even NIS plus?

    I just have mistrust in situations where Samba/CIFS get involved - I have seen things that get wierd, but against that I have also seen systems that work fine! What user account is used for the CIFS shares?

    As I am sure you are aware its the uid and gid of the file/directory that matters - especially with nfs mounts, hence the question about the common uids. Hopefully, with the NIS system that is all done and dusted - assuming that nsswitch.conf is set up for NIS before files (or that there is no duplication in names/uid between NIS and passwd/group files).
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  24. #28
  25. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2010
    Posts
    35
    Rep Power
    5
    >>> What user account is used for the CIFS shares?

    CIFS has a completely different user than the one which the script uses. Is this what you were asking for?

    As far as entries in nsswitch.conf is concerned, i saw that the order is not uniform on all platforms.

    On Linux, it's
    passwd: files nis
    group: files nis
    ...
    ...

    On HPUX, it's
    passwd: nis files
    group: nis files
    ...
    ...
  26. #29
  27. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,477
    Rep Power
    1752
    Yes, about the CIFS user - that is often the case and can often be the best way, dependant on what access they need.

    The nsswitch.conf order may not matter, make sure there is not a reason why before messing with them!
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
Page 2 of 2 First 12
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo