Weird blue outlines with Lock Transparency

[caitlinb]

ArtRage doesnt have layer clipping/masks so I duplicate a layer and use Lock Transparency to paint or shade but I've noticed thatin ArtRage 6 when I use Lock Transparency on a layer and paint on the locked layer I often get a blue outline showing at the edge of the locked section and I am not sure why.

Has anyone else gotten this? I used this all the time on ArtRage 5, and didnt have any issues then upgraded to ArtRage 6.1.2 and only sometimes this happens, but when it does happen I have to manually go around the outside of the whole area to hide the outline.

E.g A solid red circle on the layer > Lock Transparency > Airbrush any colour multiply over half the circle > The airbrushed half will have a blue outline

Is this something im doing wrong in artrage 6 perhaps ? Maybe it's me using this feature wrong or does anyone else have this happen sometimes? I've added a screenshot where an outlines showed up at the edge of the red part where I aitbrushed using a multiply effect (trying to add gentle shadows etc).

[markw]

Hello caitlinb and welcome to the ArtRage forums :)
It looks from your screenshot that you are zoomed all the way in to the painting? And along the edge of the dress there seem to be semi transparent pixels or at least pixels containing white.
Is it noticeable when viewing the whole painting at its intended size?

In general use I am unable to produce the problem you are seeing.
However, I wonder if you have performed any other actions on the painted area while Transparency Lock was active for that layer, but before using the Airbrush?
One of the quirks of AR is that if you “tidy up” as it were the edge of a painted area with say the Eraser, for example, while Transparency Lock is active, rather than remove pixel colour AR changes those areas to the same colour as the Canvas Colour, which more often than not is very light.
Some other actions that might also lead to a similar result are Cut & Paste, after using the Selection tool and softening paint edges with the Knife tool.
Subsequently when you use the Airbrush in Multiply mode any pixels that changed in colour, as above, will give the impression of a slight halo in the same colour as used by the Airbrush.

[caitlinb]

Hello caitlinb and welcome to the ArtRage forums :)
It looks from your screenshot that you are zoomed all the way in to the painting? And along the edge of the dress there seem to be semi transparent pixels or at least pixels containing white.
Is it noticeable when viewing the whole painting at its intended size?

In general use I am unable to produce the problem you are seeing.
However, I wonder if you have performed any other actions on the painted area while Transparency Lock was active for that layer, but before using the Airbrush?
One of the quirks of AR is that if you “tidy up” as it were the edge of a painted area with say the Eraser, for example, while Transparency Lock is active, rather than remove pixel colour AR changes those areas to the same colour as the Canvas Colour, which more often than not is very light.
Some other actions that might also lead to a similar result are Cut & Paste, after using the Selection tool and softening paint edges with the Knife tool.
Subsequently when you use the Airbrush in Multiply mode any pixels that changed in colour, as above, will give the impression of a slight halo in the same colour as used by the Airbrush.

Hi! Yes along the edge the pixels turn blue/transparent, and they are noticable when I zoom out to regular size which is annoying.I had not performed any other actions at that point, the rec section was drawn using the ink pen and I locked the transparency to start adding multiply on with the pen and then with the airbrush - At this point the edges became really really light. It has done this on other drawings as well, I simply ignored it until now. To fix it I have to take transparency lock off and erase the lightened edges. I avout using cut and paste for the exact reason above, the light edges are very annoying so I started using the Lock funtionality to mimick a clipping mask. Artrage 5 never did this, I am not sure what I am doing wrong with Artrage 6? Thanks for your response!

[markw]

Hello again,
Thanks for the extra info there.
And, good news, perhaps…? I have managed to replicate something similar to what you are seeing.
I was also able to reproduce this in all versions of AR.
It is being caused by the Antialiasing of paint edges. (This is automatic for most tools but in the Pen tool for example it can be turned off. However without it edges will appear stepped or jagged).
When Lock Transparency is then activated any transparent or semi transparent pixels do not darken down when being painted over by a tool using the Multiply blend mode.

So whilst this is undoubtedly something that is happening, I still find that it is not visibly noticeable in any of my finished work where I know I have used this technique too?

I found I was only able to see this effect easily if:
The layer being darkened is above a very dark layer and/or I was using a high zoom level.
I was trying to darken the semi transparent pixels to a much darker level.
And/or the canvas size had a relatively small pixel count. (This again would be analogous to zooming in to a small area of a large canvas).
(For reference I habitually use canvas sizes between:
3508 x 2482 pixels = A4 printed to 4960 x 3508 pixels = A3 printed)

Why it’s so noticeable in your example where fairly light colours have been used I’m not sure?:confused:

[caitlinb]

Hi! Thanks I'm glad its not just me! Is there a way to fix this or stop it from doing this? It is really interferring with my art unfortunately :(

The most recent occurence is today - I decided to draw over the top of an anime screenshot, and the areas I've drawn over are on a new layer, which I'd like to lock and add effects to blend it with the rest of the screenshot but my layer has the horrible edges to it whenever I try to add light, or shadow effects just as above. They're fairly small canvases, 2040 x 1000.

I'll attach screenshots below if that helps. :)

101770101769

[markw]

I think, given the relatively small size of your canvas, it might be better to come at this problem from another direction perhaps:confused:
Instead of “locking out” transparent pixels, maybe “locking in” as it were, the painted ones might give you better results.
Two options would be:
1: Select Layer Content = This will bring up the “marching ants” around the content of any given layer and nothing outside this selected area can be worked on.
The animated doted line can sometimes be a bit distracting while painting, and can be a bit more work for weaker CPUs on large canvases to track if zooming and panning the canvas a lot but on a small canvas I doubt it would be an issue.
2: New Stencil From Layer Content = This works much the same as above, but creates a protective red stencil around the immediate working area. Outside of the stencil you would still be able to paint on the layer.
Both these options can be accessed by right clicking on the desired layer in the Layers Panel and choosing them from the popup menu.
Or from the Menu bar: Tools > Layer Options > ...
If these two options are useful, you can of course set key shortcuts to activate them more quickly in AR’s Keyboard Shortcuts Preferences.

[caitlinb]

Hi - Thanks for your suggestions I gave both of these a try on the piece i sent above, unfortunately both the select layer content and the stencil are causing the same issue. I used to use select layer contents as you suggested in ArtRage 5 but the edges always became rigid, similar to this issue. This is actually why I started using Lock Transparency as it allowed me to paint over everything without any funky edges showing up but sadly this no longer works in ArtRage 6 for me. Another thing I used to do was duplicate the layer (so it is totally untouched) and immedaitely lock the transparecy so could paint over the top of it, but it's having the same results as above.

Are there any other solutions that you know of? I need a way to easily paint over the top of a layer similar to clipping mask which ArtRage doesnt have and now sadly my Lock Transparency technique no longer works. I am a little but stuck on how to get around this without having to spend extra time manually filling in all the funky edges or erasing them (both of which remove any smoothe effects I've added).

Is this a bug that could possibly be looked into or are there any other work-arounds perhaps? I'm keen to get back to painting like I used to in Artrage 5 :)

I'll pop a screenshot of my stencil attempt at adding some blue soft light just in case its something im doing wrong:


101773

[markw]

Sorry to hear it’s still not working for you yet.
I’ve tried the two alternate methods I suggested above and found they produced no visible anomalies for me along a paint edge when over painting partial pixels with the Airbrush set to multiply.
You say also that Lock Transparency used to work for you in AR5, and yet, I found the problem of combining that with semi transparent pixels to be there in all past versions of AR, given the correct circumstances?
So, are we both doing the same thing?
I wonder if you could post an exact step by step of what you are doing with settings etc…?
Also the exact size of canvas in pixels that you are working on and how it is structured?
Are you able to post an example .ptg file on a file sharing site, like Dropbox for example?
If there is a workable solution to be found for you, I fear the devil may will be in the details here!
It’s also possible our differing OS’s might be a factor here too maybe....

[Somerset]

Hi!

I'm not really sure that I understand the problem. I don't even know for sure what you mean with the "Blue outlines". But possibly, if I get it right, they are caused by unsharp edges. If the edges are blending to transparency, you will get there a colour mixed by the colour on the layer you paint on with the colour on the layer below. Blend modes of course affect the result, depending on what mode you use.

An alternative could possibly be to use a selection. To create selections that fit to your needs, you can use the Paint Area brush (Selection Tool > the right one with the brush symbol). With it you can paint a mask that will become a selection if you change the tool. But you will probably have the same problems with unsharp edges.

[caitlinb]

Hi - Selection causes the same problem sadly. Perhaps I am not describing it well, apologies!

When I draw on a layer and lock it and try to add an effect using a tool such as airbrush or ink pen the edges of the layer end up saturated and dont seem to be affected by the blend mode of the tool.

I can seem to replicate it each time not sure if these steps help:

Dark canvas background (Or any colour)
Create a new layer and draw using the Ink Pen
Lock Transparency on this layer
Airbrush on Multiply and spray some of the locked section (Opacity at 100 works well for replicating this)
Edges of the locked layer dont take the blend effect and become saturated with the selected colour instead


I appreciate all your help by the way! I have one particular piece I can share where this issue was happening with every layer inside the "Character" group. The characters were drawn using the Ink pen before I Locked the layers to add shading and I noticed an outline to each wee section. I've also added the one from the recent screenshots above. Do you have an email address I can send the Dropbox link to? I am not allowed to post URLs to the forums yet as I only joined the forums to ask about this issue whoops!

Thanks again for all your help I seriously appreciate it! :)

[markw]

Thank you for the extra info:)
I've sent you a PM with a Dropbox upload link you can use to send me the file.
Hopefully I'll find some time tomorrow to prod & poke it around!

[caitlinb]

Amazing thank you! I've popped two ptg files in there where the issue was happening I hope this helps!

I am using a Dell with OS Microsoft Windows 10 Home (Version 10.0.19041 Build 19041), meant to add this in my previous post :)

[Somerset]

Interesting issue! I reproduced your workflow, but it doesn't happen to me as far as I can see. I'm on Windows 10 Home (Version 20H2), on a Dell Computer too. Think Mark will find out what's wrong.

[caitlinb]

No problem! Interesting that its the same set up but different outcomes! As a software tester myself I understand how awkward replicating issues can be :cool:

[markw]

Thanks for the files :)
I’ve poked, prodded and experimented some and think two factors have converged to give you that visible haloing.

So,
I’ve tried this “blend mode over painting” method in all versions of AR from v3 to the current one v6. And this anomaly has always, at least in the Mac version of AR, been around, lurking in the shadows!
In low res paintings ‘Lock Transparency’ seems to consistently give the most visible haloing of semitransparent pixels on paint edges when overpainting them with a blend mode capable tool.
‘Selection from Layer’ and ‘Stencil from Layer’ may be 50% or so better, perhaps more, at reducing the haloing. But it never totally goes away if you zoom in deep enough.
(As a slight aside here, I would consider always having ‘Force Whole Pixel Alignment’ turned on in AR’s advanced preferences.
As this might help avoid accidentally making additional semitransparent pixels if cutting and pasting elements prior to overpainting them.)

Sadly I haven’t been able to find a way of entirely preventing the haloing from happening and I think the best we as users can do is to mitigate the conditions that might lead to it.

The biggest factor I think in how visible this unwanted haloing is when using this “blend mode over painting” method, is the pixel density in the overpainted areas. The lower the pixel density, the more obvious the haloing seems to become.
If you increase your starting number of pixels the proportion of semitransparent ones to solid ones in a given area drops significantly, making the haloing anomaly much harder to see, if at all, without zooming deeply into the canvas.
(And putting aside this particular issue for a moment, having more pixels making up an image will allow making any fine details in general much easier. It just gives you more to work with when it comes to those areas that need it!)

I think why this issue has perhaps not been reported before to my knowledge, could be a combination of not many people using this “blend mode over painting” technique and/or working at higher pixel density’s.
The attached image shows your sample canvas, represented in light blue, in relation to the size range of canvases I might have chosen for a similar painting. With the mid and dark blue areas being around the minimum and max size range I would be thinking of working with.

I habitually set my canvases up with printing the final image in mind (which is how I arrived at the sizes in the attached image) and how great a pixel density I will need to maintain quality of details if printed at around 16, 17 inches across and viewed close to.
But even if printing your work is not your intended end point, creating your canvas as if you were will always give you a good pixel density to work fine detail with and the option later to print the painting if desired.
If the finished image needs to be smaller it is always better to start big and reduce down than trying it the other way when it comes to maintaining quality.

I hope some of the above is helpful to you, though I know it maybe isn’t entirely what you wanted to hear:confused:

Lastly, can I say I like that Tokyo example file, nice lighting and very atmospheric of the days end!:cool:
(Sorry, any other readers of this will just have to use their imaginations at this point!:p)

101781

[caitlinb]

Although it is not quite the solution I was hoping for I didnt realise that the painting method I used was so odd, that's an interesting take-away from this! I shall try to make the canvases bigger! Since the previous post i had done another painting on a larger canvas and using a slighly different method of painting where I use lineart, so the haloing effect is lessened by the edges being hidden by the lineart. But hopefully the larger canvas will mean I can paint using my old method!

Thanks for all your time spent on this by the way, I appreciate it!