English Amiga Board


Go Back   English Amiga Board > Coders > Coders. General

 
 
Thread Tools
Old 14 October 2016, 00:31   #1
cmsj
Registered User

cmsj's Avatar
 
Join Date: Feb 2016
Location: London / UK
Posts: 165
Porting UNIX software

Hey folks

Something I'd really like to see on AmigaOS 3.x is a port of rsync, but as far as I can tell, no such thing exists.

I'd like to have a go at it, but I'm not really sure how people go about porting Linux/UNIX things to AOS. I know that ixemul.library exists, but I'm more curious about the kind of toolchains people use for this.

I had a very quick go with cahirwpz's cross compiling toolchain on Linux, but after I'd sorted out some undefined integer types, and got it all compiling, the linker failed because it couldn't find various POSIX functions like fork() execvp() and tcsetattr().

Does anyone have some hints about how this sort of work is generally done?
cmsj is offline  
Old 14 October 2016, 09:23   #2
thomas
Registered User
thomas's Avatar
 
Join Date: Jan 2002
Location: Germany
Posts: 5,942
Those functions are missing because they are not trivial to implement in AmigaOS. And that's probably the reason why a port does not yet exist. It's not a good starting point for a beginnner.

For examp├╝le for fork() you would have to create a workaround with CreateNewProc(). This usually implies a complete rewrite of the subroutine which calls fork().
thomas is offline  
Old 14 October 2016, 22:00   #3
cmsj
Registered User

cmsj's Avatar
 
Join Date: Feb 2016
Location: London / UK
Posts: 165
@thomas: Thanks for your reply (as always, you are an invaluable forum user for my programming questions!)

I stopped and thought about what you said, read the ixemul.library docs, and realised just why fork() is so hard to do on Amigas.

Fortunately for me, I managed to find a DOS port of a slightly old version of rsync, which had already been modified to run in a single process and thus not need fork().

As a result, I present:



Still a lot of work to do to get it tested and check over all the notes I made while I was quickly porting the DOS port over. Hopefully I can also roll the patch forward to at least the latest 2.x release, but hopefully the 3.x series too. Right now it won't work with a modern rsyncd that isn't forcibly downgrading its protocol version, so it can't be used to fetch things from de.aminet.net's rsyncd.

Progress though. I think these are probably the first transfers that have ever been done using rsync on an Amiga
Attached Thumbnails
Click image for larger version

Name:	IMG_8761.JPG
Views:	754
Size:	155.9 KB
ID:	50437  
cmsj is offline  
Old 15 October 2016, 01:26   #4
liviux76
Registered User

liviux76's Avatar
 
Join Date: Jan 2014
Location: BO/Italy
Posts: 711
As a Linux System Administrator I use rsync everyday... congrats on your job, cmsj!
I really hope you will be able to release a stable version of rsync for Amiga OS!
liviux76 is offline  
Old 15 October 2016, 10:54   #5
nogginthenog
Amigan

 
Join Date: Feb 2012
Location: London
Posts: 909
Quote:
Originally Posted by cmsj View Post
Progress though. I think these are probably the first transfers that have ever been done using rsync on an Amiga
I'm pretty sure I used it on an A4000 running Linux/68k :-)

Nice work. This would be great for backups. Today I periodically lha my system partition and transfer it to a SMB share.
nogginthenog is online now  
Old 15 October 2016, 11:08   #6
wXR
Registered User
 
Join Date: Mar 2009
Location: New York
Posts: 552
Fantastic work cmsj! Is your repository for this port available yet?
wXR is offline  
Old 16 October 2016, 22:05   #7
cla
dev

cla's Avatar
 
Join Date: Aug 2014
Location: Copenhagen
Age: 44
Posts: 65
Send a message via ICQ to cla
GCC Libraries

Quote:
Originally Posted by cmsj View Post
I had a very quick go with cahirwpz's cross compiling toolchain on Linux, but after I'd sorted out some undefined integer types, and got it all compiling, the linker failed because it couldn't find various POSIX functions like fork() execvp() and tcsetattr().

Does anyone have some hints about how this sort of work is generally done?
fork() is a nightmare. There is nothing similar on AmigaOS. It is a GNUish thing. POSIX Threads are more similar to AmigaOS but the signals still needs to be translated. I generally try to avoid software which uses fork() as it require a complete re-write of the threading logic.

tcsetattr() is a termios/termcap console thing. In my (short) experience I found it is best to get rid of those non Amiga native console control handling structures.

If you really need to use them the ADE toolchain includes some nice libraries: http://aminet.net/package/dev/gcc/ADE

They are ixemul based though.
cla is offline  
Old 17 October 2016, 20:51   #8
nogginthenog
Amigan

 
Join Date: Feb 2012
Location: London
Posts: 909
Just a thought: I use duplicity on a Linux box for backups. Duplicity uses librsync.
A quick scan of the code shows no fork calls.
nogginthenog is online now  
Old 18 October 2016, 02:13   #9
cmsj
Registered User

cmsj's Avatar
 
Join Date: Feb 2016
Location: London / UK
Posts: 165
Quote:
Originally Posted by nogginthenog View Post
Just a thought: I use duplicity on a Linux box for backups. Duplicity uses librsync.
A quick scan of the code shows no fork calls.
duplicity is Python though, which introduces a whole new porting problem
cmsj is offline  
Old 18 October 2016, 02:15   #10
cmsj
Registered User

cmsj's Avatar
 
Join Date: Feb 2016
Location: London / UK
Posts: 165
Quote:
Originally Posted by cla View Post
fork() is a nightmare. There is nothing similar on AmigaOS. It is a GNUish thing.
For the sake of historical accuracy, fork() was introduced in the very first version of UNIX in the early 1970s, and was inspired by earlier systems. It's not GNUish
cmsj is offline  
Old 18 October 2016, 02:16   #11
cmsj
Registered User

cmsj's Avatar
 
Join Date: Feb 2016
Location: London / UK
Posts: 165
Quote:
Originally Posted by wXR View Post
Fantastic work cmsj! Is your repository for this port available yet?
Not yet, I want to do a bit more work and testing before I start pushing the code up. It'll be going up on GitHub soon though

Edit: Specifically, I want to try and get the patch forward ported to rsync 2.6.9, which should bring compatibility with the current 3.x releases. That will actually make the release useful, by allowing it to run against modern rsync daemons that aren't being forced to downgrade their protocol version.
cmsj is offline  
Old 18 October 2016, 14:00   #12
dalek
Registered User

 
Join Date: Nov 2014
Location: NSW/Australia
Posts: 331
wow, good work!
dalek is offline  
Old 20 October 2016, 15:17   #13
cmsj
Registered User

cmsj's Avatar
 
Join Date: Feb 2016
Location: London / UK
Posts: 165
Quote:
Originally Posted by wXR View Post
Fantastic work cmsj! Is your repository for this port available yet?
Hey, so I've just pushed up the port of 2.5.5 to aminet (util/misc/rsync-2.5.5_bin.lha and util/misc/rsync-2.5.5_src.lha) and the git repo is https://github.com/cmsj/rsync/

Unfortunately I ran out of time in trying to forward port the Amiga changes to 2.6.9, but I did manage to clean them up quite a lot, in terms of reducing the overall size of the diff vs upstream rsync.
If anyone would like to pitch in and help, I would be very very grateful, I'm not sure how much more effort I can put into this.
cmsj is offline  
Old 26 October 2016, 21:02   #14
cla
dev

cla's Avatar
 
Join Date: Aug 2014
Location: Copenhagen
Age: 44
Posts: 65
Send a message via ICQ to cla
I will give it try.

And sorry about the inaccurate description of fork() earlier in the thread. I accidentally mixed up the threading models with something else.
cla is offline  
Old 08 March 2017, 11:59   #15
ciVic
Registered User

 
Join Date: Oct 2013
Location: Germany
Posts: 30
I've set up an rsync Server on my Windows host und try your port in WinUAE:

rsync 127.0.0.1::share/

I also started the Server with --protocol=30. But all I get is an error that the funtction is not implemented in clientserver.c 97

Last edited by ciVic; 08 March 2017 at 17:28.
ciVic is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Possible to run Amiga Unix on WinUAE? Zombie13 New to Emulation or Amiga scene 5 24 January 2019 15:26
Amiga Unix manuals Amiga_CDTV support.Amix 4 06 September 2015 07:08
xxxBSD/Unix/Linux Doc Mindie request.UAE Wishlist 2 16 December 2004 22:47
Hardfile support for Unix version Seti support.WinUAE 1 29 November 2003 18:29
Unix UAE FromWithin support.WinUAE 3 13 November 2003 11:18

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 19:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Page generated in 0.10249 seconds with 16 queries