Wednesday, April 23, 2014

Sync Synology with Dropbox and Google Drive

With DSM 5.0 Synology introduced a feature a lot of people have been asking for - the possibility to sync or use your Synology with Dropbox or Google Drive.

What a cool way to:

1) transfer files from whatever to your Synology! Dropbox or Google Drive is compatible with almost every platform out there, be it Mac, Windows, IPhone, iPad, Android or Windows phone. As soon as you upload for example a picture from your iPhone to Dropbox, it will automatically be synced to your Synology.

2) Share! If you have a shared folder in Dropbox, by adding a file to that folder on your Synology, you are sending it to people you have that particular folder shared with.

3) Cloud Backup! Google has taken a decision in March 2014 to slash prices of google drive. I currently use Amazon Glacier on my Synology to backup a lot of information - mostly Pictures and videos. As per my blog post on why I bought the Synology it had an automatic upload system to Glacier and my monthly cost for an online backup of a lot of information (TBs) is only around 5$.

However Amazin Glacier has disadvantages. It had no dedicated program to upload / download info. Files are stores without filenames, so the program you use needs to solve this. And finally it is sloooowwww. A request to take something out of Glacier takes approx 4 hours to start the download, and is extremely slow to send. Good for cheap backup of files you do not need in a rush or to modify often, but not for much more.

So, back to google drive. With the price slash, google is turning storage into a commodity. For a whopping 1TB of info, google charges only 9.99$ per month! If you need less, 100Gb for only 1.99USD And we all know the advantages of google drive - it's fast, secure, files are just there, it even keeps previous copies.

 

Any limitations?

I only saw:

1) you can only synchronize your complete Dropbox or Google Drive folder. You can't select a set of subfolders. It's either everything or nothing!

2) for the Mac you might be wasting Synology space with multiple backups. If you use Time Machine on your Mac with Synology with your backup disk and you have Dropbox installed it is backing up th 2-10Gbs of info via time capsule to the Synology and if you install this sync it repeats the info again. The solution? Tell Time Capsule to exclude the Dropbox folder from the backup.

 

How to install?

You need:

- A Synology installed with DSM 5 and internet access. If unsure on what version you have, It should show on the bottom left DSM 5.0 on the main window.

- Basic knowledge around the Synology interface

- A Google Drive or Dropbox account

 

With the above checked, let's start!

1) Log in to your Synology and head over to the package center.

2) In the upper right corner of the package center is a search box. Search for Cloud.

3) 3 options should appear - Cloud station, Cloud Station Client ( both for making your personal cloud) and Cloud Sync. Press Cloud Sync for more information ( as image below ) or directly press install.

4) After installing when you enter the base menu a new option appears called Cloud Sync. Open the application.

Synology DSM 5.0 main menu
Synology DSM 5.0 main menu

5) When you enter the program for the first time, it will automatically open up the connection wizard. Select the service you want to configure. We will start with Dropbox. Select it and press next.

Synology Cloud Sync Wizard
Synology Cloud Sync Wizard

6) a pop-up window will appear asking you to allow cloud sync to access your Dropbox. If you are not already logged onto Dropbox in your browser, you will need to login before the allow screen appears.

Synology Cloud Sync
7) The pop-up will disappear, and the wizard will continue. The next step is to select where you want the Dropbox folder synced. Remember it will be the whole Dropbox, including every subfolders and file. On my Synology I created a separate shared folder just for this with it's own permissions.
Synology Cloud Sync
Synology Cloud Sync wizard folder

7) After pressing Apply, the app will start syncing automatically. Anything changed in your Dropbox will mirror in the folder you selected, like any other Dropbox app. Anything you change in the Synology Dropbox folder will change everywhere else as well.

Synology Cloud Sync Main Window
Synology Cloud Sync Main Window

In the top corner of your Synology main page a cloud icon will appear. It will reflect the status of your sync. A green tick is synced, a blue sync in progress. Depending how many files you have it will take more or less time for the first sync.

 

Synology main screen with iCloud sync
Synology main screen with iCloud sync

And Google Drive? Can I have both at the same time?

Yes, both can be installed. The process is exactly the same.

1) Open Cloud Sync. Press the + button on the bottom left to start the wizard.

Synology cloud sync main window
Synology cloud sync main window

2) Select Google Drive on the first wizard screen and press next

Synology Cloud Sync wizard
Synology Cloud Sync wizard

3) A pop-up will appear to validate the access with Google. Press accept. If you are not logged into google on the browser, a login screen will appear first.

Synology google accept
Synology google accept

4) select the folder where you want to Sync. Press apply and...

5) The sync will start!

Synology cloud sync main window
Synology cloud sync main window

Wait until both accounts turn green. All files have been downloaded both for Dropbox and Google drive.



As a personal backup options for your files, I would recommend google drive given the 1Tb pricing being so cheap. For sharing and having folders synced with friends / family, Dropbox still wins for me.


Enjoy!

 

Friday, April 18, 2014

Compress HD video for free from 10Gb to 1GB on a Mac

After using my automatic duplicate and big file finder on my Synology (see CharlieTechLife blog to set-up) I saw that I had several HD videos weighing a whopping 12 to 15Gb in .ts format. This means over 100GBs in vĂ­deos I see maybe once per year.
So I started investigating how I could compress those videos into a more reasonable size. My objective is to have each file reduced to approx. 1Gb with minimal quality loss and be compatible with playback on PS3, IOS devices, Windows, etc..., so I need a compression codec (divx, xvid, x264) and a container (.avi, .divx, .mkv) compatible with these devices.
On the Mac there are two software programs that are FREE that allow you to do just this. One is Handbrake, a free video transcoder that works extremely well. The second is Mediacalc, a bitrate calculator - it's there to ensure your video ends up being the right size.
After you download them following the links above, let's get started!
Fire up Handbrake. The interface is pretty straightforward.
Handbrake interface
Press source and select the video you want to compress. Under title you will see details of the video. Before our next step jot down the duration ( in my example it's.00:41:57). We will use this in a minute to ensure the compressed file is 1Gb.
As my Output setting options:
- Format: MP4 file. The mp4 format is playable across most platforms I need (iPad, IPhone, PS3, Xbox...)
- Large file size: uncheck. This is to enable 64bit - it allows files bigger than 4Gb for nonFAT32 systems. Will not affect any compression beneath this size, and if checked can limit compatibility
- Web Optimized: Checked. This places the container header in the beginning of the file. It will make the video start faster if streaming.
Under the video tab:
- Video Codec: H.264 (x.264). This codes into MPEG-4 part 10, also known as H.264/AVC. It gives much better quality and lower file sizes than the normal MPEG-4 / VP3 encoders and all modern devices like I want to use will support this format. (iPhone, iPad, BluRay Players, PS3, XBox360, etc....)
Framerate: Same as source. I want to keep the video quality as close to the original as possible. As such I keep the same frame rate as per the original one.
Variable framerate: ticked. I keep this as to imitate the original.
Video quality: Average bitrate. Here there is no preview available to see how the video comes out in terms of quality or size, be it using constant quality or average bitrate option. As my main limitation is size, if I wanted to use the constant quality option I would need to try a number on the scale, compress, see file size, try another number and compress again. So for a specific size, I use average bitrate.
2-pass encoding: ticked. Takes longer (almost double the time), but results are worth it.
Mediacalc:
MediaCalc

We will just use the first Calculation part of the application. We fill in:
- Media size: 1Gb. This is the final size we want.
- Video Duration: 41.57. Here we put the exact size of our original video that we jotted down from handbrake as per above.
- Audio bitrate: 160kbps. My original file has this bitrate. The program needs this number as to see what is the total file size of audio+video. The more space occupied by audio, the less we have for video.
- Overhead: 2% this is how much space is used by the mp4 container. I haven't found the exact number, but 2% seems to keep the file close to what I intend it to be.
The number that appears under Video Datarate is the one you want to use in handbrake (3135.981 Kbps). Make sure you have the result in Kbps.

Back to handbrake. After setting the Average bitrate, we get into the H.264 codec options. Here we can make a huge difference on the end result in terms of look&feel of the video. Of course there is a trade off: the better looking the video the more time is needed to encode it, and a better processor needed to decode it. For example an iPad 1 will struggle to decode the video with the settings I will use here.
Handbrake x264 options
Use x264 Advanced Options Panel: unchecked as we will use this more basic interface
X264 Preset: Very Slow. Here the options on the scale go from Ultra fast to placebo. Ultra fast as the name indicates is a extremely quick encoder, but at the cost of video quality. For example same colors tend to be blocks vs. HD resolution. This is the setting I find affects video quality the most, the slower you encode the better the end result will be. Placebo speed is overkill, giving only slightly better results than very slow at a extreme time cost.
X264 Tune: Film. Select the type that is closest to the content you are encoding. Possibilities are:
Handbrake x264 tune
  • Film: Filmed footage (I.e. Real life movie) footage, not Cartoons or Anime. If in doubt, use this one by default.
  • Animation: Use for hand-drawn animated content.
  • Grain: Use for very grainy or old content.
  • Still Image: Use for Photo slideshows or presentations where there is very little moving content.
  • Psnr/sslm: used for testing. Do not use.
  • Zerolatency: Optimise for faster decoding, Useful for low power devices that struggle to decode the video like ipad 1.
  • Fast decode: uncheck
H.264 Profile: High. There are 4 options here:
handbrake H.264 profile
  • Auto: will automatically set the profile based on all the options that have been selected.
  • Baseline: Baseline is the most basic form of encoding. While decoding is much easier, it may also require much higher bit-rates to maintain the same level of quality.
  • Main: The middle ground. Most modern / current devices will support this profile.
  • High Profile: For best quality and filesize at the expense of CPU time in both decode and encode.
If after encoding the video stutters on the playback device, try lowering this from high profile to main or baseline.
H.264 Level: 4.0. This has to do with frame rate and resolution. Leave with handbrake default.
Change to the Audio tab in handbrake:
Handbrake audio tab
As I don't want to re-encode the audio for this video we choose "Audio Passthru" from the drop down list.

Go back to video and check that everything is OK.
Handbrake encode check
If yes, press add to queue. If you want to encode more than one file, repeat the process after selecting source for the next file. When you have everything in the queue, press start. Leave the computer for a couple hours to encode... And voila!!!!! Compressed movie done!

Friday, April 11, 2014

Synology tip: How to find duplicate files or large files automatically

I love my Synology, as I use it for many things and it just works. From media server, to file archive, to backup, to photo server, to surveillance... It has many possibilities. If you add high capacity drives (I have 2 x 3 TB), it can hold quite a lot of information.

Lately I've been centralizing all digital information on the Synology, mainly because I used to have info on multiple computers, external hard drives, backups of backups, etc... and wanted to have everything in one place. After copying and organizing a bit, I found out that I had multiple copies of same files, pictures, videos, etc...

But how to check and erase only duplicates?

 

I started by using Gemini, a duplicate finding software for MacOS. However the search needs to be done via network (executed on the Mac) and takes forever to generate a report. Wouldn't it be great if Synology could do this without a computer involved?

Luckily, Synology thought of a way to do this automatically by the DSM. Not only that but it can automatically generate duplicate reports on a regular basis and email them to you. The email contains a link which takes you to a webpage where you can delete the duplicate file. EASY!

How to set it up:

Enter your Synology via the desktop DSM ( or simply put login to your Synology ). If you have DSM 5, you should see something like this:

Synology DSM 5 main screen
Charlietechlife.blogspot.com

Open the control panel and under "System" go to "Info center"

Information on your Synology product will appear. On the top there is a tab named "Report". Click on it

Press "Create" to start the report wizard

In the next screen give a identifiable name to your report. I called mine "Duplicates". Also add an email so that your Synology can email you a direct link when the report is ready.
 

After pressing next, the screen will contain several options. Check Only "Duplicate File Candidates" and press next.

 
 

The next screen is about scheduling, when do you want Synology to generate the report. I have it once per week every Monday at 00:00. To note this is when it starts generating the report, not when it finishes or sends you the email. In the case you want to verify duplicated (next screen), and depending on the quantity of files you have it make take a while.

After pressing next, you can check "Find confirmed duplicates", which makes 100% sure the files shown are duplicates. I definitely recommend this option. Here you can also choose to check every files in the Synology or only specific folders.

In the next screen verify your options and press "Apply". And there it is! After a some time you will get your first duplicate report.

 

And within it you can erase the files you don't want to keep!

 

Thursday, April 10, 2014

How to delete or show only archived emails in Gmail


Apparently over the years I didn't configure some of my devices correctly to interact with gmail. If I blame Blackberry, Outlook, Mac Mail, Gmail app for blackberry, IOS 4, IOS 5, the iPhone 3GS, the iPad 1, iPad Air, iPhone 4s or iPhone 5s, I don't know.
(Mental note.. way too many gadgets passed through my hands!)

Why do I think incorrectly? Because instead of deleted emails like I wanted to when I trash them, some of these apps / devices (not sure which ones) would archive them. Meaning they are still occupying space in gmail and are searchable. Google seems to like that you never erase anything and it is the default behaviour in most interfaces with their mail system.

The issue? There seems to be no easy way of finding this archive and deleting it. Gmails' archive is simply an email with no labels. A label is how gmail classifies email into virtual folders. How to see if you have the same issue?

On the left of the gmail web interface you have a list of all your labels ( Inbox, Sent mail, Drafts, etc... ). Scroll down until you see "All mail"

Gmail all mail

After pressing, you should see a list of emails and next to the subject the label they belong to (Inbox, etc..). If you see no label as the email "es espectacular" below.... It's because it is archived and occupying space.

Gmail all mail

 

However gmail doesn't have a way of filtering these archived messages as to delete them. To do so, enter in the gmail search box ( on the top ) the following filter search:

has:nouserlabels -in:Inbox -in:Sent -in:Draft -in:Chat

A list of archived messages should appear and you can erase all of them. Double check none has a label before erasing....