自定义Ranger插件 Zeppelin集成Ranger方案 （非LDAP模式）
序：
最近项目需求要用Ranger把Zeppelin管起来，之前聊过使用LDAP的模式，这次我们说说不使用LDAP，通过开发Ranger插件的方式将Zeppelin和Ranger继承起来
目录
序：
一、Zeppelin权限部分分析
1、zeppelin-note权限分析
2、zeppelin-shiro权限分析
二、shiro权限原理分析及Ranger原理分析
1、shiro认证授权原理
2、Ranger认证授权原理
三、自定义Ranger插件
1、编写shiroService
2、编写资源json文件
3、编写Ranger授权类
四、自定义Shiro拦截器（RangerFilter）
五、测试
一、Zeppelin权限部分分析
由官方文档可知，zeppelin权限是使用shiro接管的，认证、授权默认使用的shiro的textRealm，同时还支持LDAPRealm、PamRealm、JDBCRealm等其他Realm。
Zeppelin中的权限有两个大模块，一方面是note粒度的权限，另一方面是菜单权限粒度，解释器配置页面的访问权限、版本控制页面的访问权限以及其他一些页面url的拦截。
1、zeppelin-note权限分析
对于note粒度的权限，划分为可读可写可执行，配置的方式目前有两种：页面配置和RESTApi设置，如下图：
源码中实现过程是在org.apache.zeppelin.service.NotebookService类中，所有的note操作都要经过org.apache.zeppelin.service.NotebookService.checkPermission()方法来鉴权
enum Permission {READER,WRITER,RUNNER,OWNER,}/*** Return null when it is allowed, otherwise return the error message which could be* propagated to frontend** @param noteId* @param context* @param permission* @param op* @return*/private boolean checkPermission(String noteId,Permission permission,Message.OP op,ServiceContext context,ServiceCallback callback) throws IOException {boolean isAllowed = false;Set allowed = null;switch (permission) {case READER:isAllowed = authorizationService.isReader(noteId, context.getUserAndRoles());allowed = authorizationService.getReaders(noteId);break;case WRITER:isAllowed = authorizationService.isWriter(noteId, context.getUserAndRoles());allowed = authorizationService.getWriters(noteId);break;case RUNNER:isAllowed = authorizationService.isRunner(noteId, context.getUserAndRoles());allowed = authorizationService.getRunners(noteId);break;case OWNER:isAllowed = authorizationService.isOwner(noteId, context.getUserAndRoles());allowed = authorizationService.getOwners(noteId);break;}if (isAllowed) {return true;} else {String errorMsg = "Insufficient privileges to " + permission + " note.\n" +"Allowed users or roles: " + allowed + "\n" + "But the user " +context.getAutheInfo().getUser() + " belongs to: " + context.getUserAndRoles();callback.onFailure(new ForbiddenException(errorMsg), context);return false;}} 2、zeppelin-shiro权限分析
zeppelin默认使用shiro.ini的方式来配置权限，它还支持从LDAP和其他Realm同步用户 zeppelin集成LDAP https://blog.csdn.net/EdwardWong_/article/details/115401467
我们在shiro中配置好的url，最终都会走org.apache.shiro.web.filter.authz.RolesAuthorizationFilter.isAccessAllowed()方法来处理，处理原理此处不细说，如果我们想使用Ranger来处理权限，此处是着手之处
package org.apache.shiro.web.filter.authz;import java.io.IOException;
import java.util.Set;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import org.apache.shiro.subject.Subject;
import org.apache.shiro.util.CollectionUtils;// shiro 源码
public class RolesAuthorizationFilter extends AuthorizationFilter {public RolesAuthorizationFilter() {}public boolean isAccessAllowed(ServletRequest request, ServletResponse response, Object mappedValue) throws IOException {Subject subject = this.getSubject(request, response);String[] rolesArray = (String[])mappedValue;if (rolesArray != null && rolesArray.length != 0) {Set roles = CollectionUtils.asSet(rolesArray);return subject.hasAllRoles(roles);} else {return true;}}
}
二、shiro权限原理分析及Ranger原理分析
1、shiro认证授权原理
// Todo...
2、Ranger认证授权原理
// Todo...
对于note粒度的权限，如果我们使用Ranger来鉴权，对zeppelin源码入侵性过大，后续zeppelin官方的升级适配和其他一些问题过于繁琐，所以再三思考过后决定先实现菜单权限层面的Ranger管控
三、自定义Ranger插件
官方给出了一些案例，阅读完后感觉一言难尽，了解到一些大概的流程，但是对于开发自定义插件帮助实在有限，于是一边借鉴其他官方Ranger插件的代码，一遍捋Ranger源码，最后终于开发成功。
ranger插件包层次如下
1、编写shiroService
pom文件如下：
ranger org.apache.ranger 2.0.0 4.0.0 ranger-shiro-plugin org.apache.shiro shiro-core 1.7.0 org.apache.shiro shiro-web 1.7.0 org.apache.shiro shiro-config-core 1.7.0 org.apache.ranger ranger-plugins-common 2.0.0 org.apache.ranger ranger-plugins-audit 2.0.0 org.apache.ranger credentialbuilder 2.0.0 org.apache.maven.plugins maven-shade-plugin 2.3 package shade 首先编写RangerService类，需要继承RangerBaseService类，并实现init()、validateConfig()、lookupResource方法。validateConfig()方法是对参数的验证方法，会对在RangerWeb界面上配置的zeppelin参数进行认证。lookResource方法是RangerWeb的一个自动补全参数方法，代码如下
package org.apache.ranger.services.shiro;import org.apache.ranger.plugin.client.BaseClient;
import org.apache.ranger.plugin.model.RangerService;
import org.apache.ranger.plugin.model.RangerServiceDef;
import org.apache.ranger.plugin.service.RangerBaseService;
import org.apache.ranger.plugin.service.ResourceLookupContext;import java.util.HashMap;
import java.util.List;
import java.util.Map;/*** @author Edwards Wang* @version 2021/4/13 下午5:39* describe:*/
public class RangerServiceShiroTest extends RangerBaseService {@Overridepublic void init(RangerServiceDef serviceDef, RangerService service) {super.init(serviceDef, service);}@Overridepublic Map validateConfig() throws Exception {String errMes = "Failed,cannot connext to the zeppelin Service";String successMes = "Connection Success";Map map = new HashMap<>();if (!configs.containsKey("interpreterPage")){BaseClient.generateResponseDataMap(false,errMes,errMes,null,null,map);}else {BaseClient.generateResponseDataMap(true,successMes,successMes,null,null,map);}return map;}@Overridepublic List lookupResource(ResourceLookupContext context) throws Exception {return null;}}
2、编写资源json文件
我们要对我们的插件有一个描述文件，文件要放在agents-common/src/main/resources/service-defs/路径下
首先是对插件的描述，需要有id、name、类路径、标签和描述
"id":29,"name": "shiroTest2","implClass": "org.apache.ranger.services.shiro.RangerServiceShiroTest","label": "SHIRO Repository","description": "SHIRO Repository","resources":然后是对我们插件要管控的资源的描述
{"itemId": 1,"name": "path","type": "path","level": 10,"parent": "","mandatory": true,"lookupSupported": true,"recursiveSupported": true,"excludesSupported": false,"matcher": "org.apache.ranger.plugin.resourcematcher.RangerPathResourceMatcher","matcherOptions": { "wildCard":true, "ignoreCase":false },"validationRegEx":"","validationMessage": "","uiHint":"","label": "url","description": "url"}然后是对权限的设置，这里只开启了一个权限get（此get和http请求无关，只是是否可以访问）
{"itemId": 1,"name": "get","label": "get"}完整的文件内容如下
{"id":29,"name": "shiroTest2","implClass": "org.apache.ranger.services.shiro.RangerServiceShiroTest","label": "SHIRO Repository","description": "SHIRO Repository","resources":[{"itemId": 1,"name": "path","type": "path","level": 10,"parent": "","mandatory": true,"lookupSupported": true,"recursiveSupported": true,"excludesSupported": false,"matcher": "org.apache.ranger.plugin.resourcematcher.RangerPathResourceMatcher","matcherOptions": { "wildCard":true, "ignoreCase":false },"validationRegEx":"","validationMessage": "","uiHint":"","label": "url","description": "url"}],"accessTypes":[{"itemId": 1,"name": "get","label": "get"}],"configs":[{"itemId": 1,"name": "interpreterPage","type": "string","subType": "","mandatory": true,"validationRegEx":"","validationMessage": "","uiHint":"","label": "interpreterPage"}]
}
在完成service和资源文件后，需要执行命令将我们的插件加载进Ranger，命令如下：
curl -u admin:admin -X POST -H "Accept: application/json" -H "Content-Type: application/json" –d @ranger-servicedef-shiro.json 'http://localhost:6080/service/public/v2/api/servicedef'然后需要重启RangerAdmin，接下来打开RangerWeb界面就能看到我们的插件了
3、编写Ranger授权类
我们的RangerShiroAuthorizer要继承shiro的org.apache.shiro.web.filter.authz.AuthorizationFilter类，并实现isAccessAllowed方法
package org.apache.ranger.authorization.shiro.authorizer;import org.apache.ranger.plugin.audit.RangerDefaultAuditHandler;
import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
import org.apache.ranger.plugin.policyengine.RangerAccessResourceImpl;
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
import org.apache.ranger.plugin.service.RangerBasePlugin;
import org.apache.shiro.subject.Subject;
import org.apache.shiro.web.filter.authz.AuthorizationFilter;import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;/*** @author Edwards Wang* @version 2021/4/13 下午2:16* describe:*/
public class RangerShiroAuthorizer extends AuthorizationFilter {public RangerShiroAuthorizer() {}@Overridepublic boolean isAccessAllowed(ServletRequest request, ServletResponse response, Object mappedValue) throws Exception {Subject subject = getSubject(request,response);// 获取用户String userName = (String) subject.getPrincipal();HttpServletRequest httpServletRequest = (HttpServletRequest) request;// 获取要拦截的urlStringBuffer url = httpServletRequest.getRequestURL();RangerAccessResourceImpl resource = new RangerAccessResourceImpl();resource.setValue("path",url.toString());RangerAccessRequestImpl rangerAccessRequest = new RangerAccessRequestImpl(resource,"get",userName,null);// 此处的名字要和json文件中定义的name插件名一致RangerBasePlugin rangerBasePlugin = new RangerBasePlugin("shiroTest2", "shiroTest2");rangerBasePlugin.setResultProcessor(new RangerDefaultAuditHandler());rangerBasePlugin.init();// 将权限交由Ranger来审核，返回boolean的值再交给shiroRangerAccessResult result = rangerBasePlugin.isAccessAllowed(rangerAccessRequest);return result != null && result.getIsAllowed();}
}
4、编写audit文件和security文件，我们要将这两个文件放在插件包的resource路径下：
首先是ranger-shiroTest2-audit.xml
xasecure.audit.destination.db false xasecure.audit.destination.db.jdbc.driver com.mysql.jdbc.Driver xasecure.audit.destination.db.jdbc.url jdbc:mysql://localhost/ranger_audit xasecure.audit.destination.db.password rangerlogger xasecure.audit.destination.db.user rangerlogger xasecure.audit.destination.db.batch.filespool.dir /tmp/audit/db/spool xasecure.audit.destination.hdfs false xasecure.audit.destination.hdfs.dir hdfs://localhost:8020/ranger/audit xasecure.audit.destination.hdfs.batch.filespool.dir /tmp/audit/hdfs/spool xasecure.audit.destination.log4j true xasecure.audit.destination.log4j.logger ranger_audit_logger 然后是ranger-shiroTest2-security.xml
有一个属性，需要特别注意。 这就是名为ranger.plugin.httpservice.service.name的属性。 此属性的值必须与您在Ranger UI中使用的服务名称相同。
ranger.plugin.shiroTest2.policy.rest.url http://192.168.101.48:6080 URL to Ranger Admin ranger.plugin.shiroTest2.service.name MyService Name of the Ranger service containing policies for this shiroTest2 instance ranger.plugin.shiroTest2.policy.source.impl org.apache.ranger.admin.client.RangerAdminRESTClient Class to retrieve policies from the source ranger.plugin.shiroTest2.policy.rest.ssl.config.file ranger-policymgr-ssl.xml Path to the file containing SSL details to contact Ranger Admin ranger.plugin.shiroTest2.policy.pollIntervalMs 30000 How often to poll for changes in policies? ranger.plugin.shiroTest2.policy.cache.dir /tmp Directory where Ranger policies are cached after successful retrieval from the source ranger.plugin.shiroTest2.policy.rest.client.connection.timeoutMs 120000 RangerRestClient Connection Timeout in Milli Seconds ranger.plugin.shiroTest2.policy.rest.client.read.timeoutMs 30000 RangerRestClient read Timeout in Milli Seconds 然后打包。接下来开始写shiro拦截器部分
四、自定义Shiro拦截器（RangerFilter）
在zeppelin的zeppelin-server/src/main/java/org/apache/zeppelin下新建一个filter包，在filter包下新建一个shiro拦截器：RangerFilter，继承org.apache.shiro.web.filter.authz.AuthorizationFilter方法，实现isAccessAllowed方法，在方法里直接调用我们ranger插件包内的isAccessAllowed方法
package org.apache.zeppelin.filter;import org.apache.ranger.authorization.shiro.authorizer.RangerShiroAuthorizer;
import org.apache.shiro.web.filter.authz.AuthorizationFilter;import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;/*** @author Edwards Wang* @version 2021/4/20 下午2:30* describe:*/
public class RangerFilter extends AuthorizationFilter {@Overrideprotected boolean isAccessAllowed(ServletRequest request, ServletResponse response, Object mappedValue) throws Exception {return new RangerShiroAuthorizer().isAccessAllowed(request,response,mappedValue);}
}
然后修改shiro.ini文件，在[main]模块下新增条目
[main]
rangerFilter = org.apache.zeppelin.filter.RangerFilter在[url]模块下修改条目，此处只以解释器页面为例
[urls]
/api/version = anon
/api/cluster/address = anon
/api/interpreter/setting/restart/** = authc
/api/interpreter/** = authc, rangerFilter完整的shiro.ini文件如下
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#[users]
# List of users with their password allowed to access Zeppelin.
# To use a different strategy (LDAP / Database / ...) check the shiro doc at http://shiro.apache.org/configuration.html#Configuration-INISections
# To enable admin user, uncomment the following line and set an appropriate password.
#admin = password1, admin
user1 = password2, role1, role2
user2 = password3, role3
user3 = password4, role2# Sample LDAP configuration, for user Authentication, currently tested for single Realm
[main]
### A sample for configuring Active Directory Realm
#activeDirectoryRealm = org.apache.zeppelin.realm.ActiveDirectoryGroupRealm
#activeDirectoryRealm.systemUsername = userNameA#use either systemPassword or hadoopSecurityCredentialPath, more details in http://zeppelin.apache.org/docs/latest/security/shiroauthentication.html
#activeDirectoryRealm.systemPassword = passwordA
#activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/zeppelin.jceks
#activeDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COM
#activeDirectoryRealm.url = ldap://ldap.test.com:389
#activeDirectoryRealm.groupRolesMap = "CN=admin,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"admin","CN=finance,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"finance","CN=hr,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"hr"
#activeDirectoryRealm.authorizationCachingEnabled = false### A sample for configuring LDAP Directory Realm
#ldapRealm = org.apache.zeppelin.realm.LdapGroupRealm
## search base for ldap groups (only relevant for LdapGroupRealm):
#ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COM
#ldapRealm.contextFactory.url = ldap://ldap.test.com:389
#ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COM
#ldapRealm.contextFactory.authenticationMechanism = simple### A sample PAM configuration
#pamRealm=org.apache.zeppelin.realm.PamRealm
#pamRealm.service=sshd### A sample for configuring ZeppelinHub Realm
#zeppelinHubRealm = org.apache.zeppelin.realm.ZeppelinHubRealm
## Url of ZeppelinHub
#zeppelinHubRealm.zeppelinhubUrl = https://www.zeppelinhub.com
#securityManager.realms = $zeppelinHubRealm## A same for configuring Knox SSO Realm
#knoxJwtRealm = org.apache.zeppelin.realm.jwt.KnoxJwtRealm
#knoxJwtRealm.providerUrl = https://domain.example.com/
#knoxJwtRealm.login = gateway/knoxsso/knoxauth/login.html
#knoxJwtRealm.logout = gateway/knoxssout/api/v1/webssout
#knoxJwtRealm.logoutAPI = true
#knoxJwtRealm.redirectParam = originalUrl
#knoxJwtRealm.cookieName = hadoop-jwt
#knoxJwtRealm.publicKeyPath = /etc/zeppelin/conf/knox-sso.pem
#
#knoxJwtRealm.groupPrincipalMapping = group.principal.mapping
#knoxJwtRealm.principalMapping = principal.mapping
#authc = org.apache.zeppelin.realm.jwt.KnoxAuthenticationFilter
rangerFilter = org.apache.zeppelin.filter.RangerFiltersessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager### If caching of user is required then uncomment below lines
#cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
#securityManager.cacheManager = $cacheManager### Enables 'HttpOnly' flag in Zeppelin cookies
cookie = org.apache.shiro.web.servlet.SimpleCookie
cookie.name = JSESSIONID
cookie.httpOnly = true
### Uncomment the below line only when Zeppelin is running over HTTPS
#cookie.secure = true
sessionManager.sessionIdCookie = $cookiesecurityManager.sessionManager = $sessionManager
# 86,400,000 milliseconds = 24 hour
securityManager.sessionManager.globalSessionTimeout = 86400000
shiro.loginUrl = /api/login[roles]
role1 = *
role2 = *
role3 = *
admin = *[urls]
# This section is used for url-based security. For details see the shiro.ini documentation.
#
# You can secure interpreter, configuration and credential information by urls.
# Comment or uncomment the below urls that you want to hide:
# anon means the access is anonymous.
# authc means form based auth Security.
#
# IMPORTANT: Order matters: URL path expressions are evaluated against an incoming request
# in the order they are defined and the FIRST MATCH WINS.
#
# To allow anonymous access to all but the stated urls,
# uncomment the line second last line (/** = anon) and comment the last line (/** = authc)
#
/api/version = anon
/api/cluster/address = anon
# Allow all authenticated users to restart interpreters on a notebook page.
# Comment out the following line if you would like to authorize only admin users to restart interpreters.
/api/interpreter/setting/restart/** = authc
/api/interpreter/** = authc, rangerFilter
/api/notebook-repositories/** = authc, roles[admin]
/api/configurations/** = authc, roles[admin]
/api/credential/** = authc, roles[admin]
/api/admin/** = authc, roles[admin]
#/** = anon
/** = authc
修改完毕，将ranger和ranger插件的包放到lib路径下重启zeppelin
五、测试
首先打开zeppelin的界面，使用默认的user1用户登录，访问interpreter界面，会发现报错说没有权限
然后我们进入RangerWeb界面配置权限，首先配置插件
然后配置策略，将url和用户角色权限加上
点击add或save，然后重新回到zeppelin界面，这回就可以正常访问解释器页面了
大功告成！！
有任何问题可以评论留言，开发过程中有一些坑，大家遇到我踩过的可以帮大家解答一下
本文来自互联网用户投稿，文章观点仅代表作者本人，不代表本站立场，不承担相关法律责任。如若转载，请注明出处。 如若内容造成侵权/违法违规/事实不符，请点击【内容举报】进行投诉反馈！